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Preface 


The  Army  has  been  making  policy,  organizational,  and  information 
system  changes  to  support  Total  Life  Cycle  Systems  Management,  the  “cradle- 
to-grave”  management  of  weapon  and  materiel  systems.  Recent  reports, 
however,  have  described  cases  of  critical  life  cycle  management  decisions  and 
supporting  analyses  being  hindered  by  problems  with  life  cycle  sustainment 
(LCS)  data,  i.e.,  information  about  the  operations,  support,  and/or  disposal  of 
Army  equipment.  This  document  describes  a  RAND  Arroyo  Center  study 
conducted  to  provide  the  Army  with  a  comprehensive  assessment  of  LCS  data 
currently  available  in  Standard  Army  Management  Information  Systems 
(STAMIS).  Findings  suggest  that  a  range  of  data  access,  quality,  and  breadth 
issues  need  to  be  addressed  to  ensure  that  rigorous  analyses  can  be  conducted  in 
support  of  critical  LCM  decisions.  Recommendations  include  a  combination  of 
potential  Army  policy  revisions,  information  system  design  changes,  and  steps 
to  improve  execution  of  sustainment  data  policies. 

Results  of  this  study  should  be  of  interest  to  Army  managers,  analysts,  and 
information  system  developers  in  the  acquisition  and  logistics  communities. 
This  research  was  sponsored  by  the  Deputy  Chief  of  Staff,  G-4,  Department  of 
the  Army,  Resource  Integration  Directorate  and  was  conducted  in  RAND 
Arroyo  Center’s  Military  Logistics  Program.  RAND  Arroyo  Center,  part  of  the 
RAND  Corporation,  is  a  federally  funded  research  and  development  center 
sponsored  by  the  United  States  Army.  Questions  and  comments  regarding  this 
research  are  welcome  and  should  be  directed  to  the  director  of  the  Military 
Logistics  Program,  Ken  Girardini,  at  girardin@rand.org. 

The  Project  Unique  Identification  Code  (PUIC)  for  the  project  that 
produced  this  document  is  DALOC10515. 
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For  more  information  on  RAND  Arroyo  Center,  contact  the  Director  of 
Operations  (telephone  310-393-0411,  extension  6419;  fax  310-451-6952; 
email  Marcy_Agmon@rand.org),  or  visit  Arroyo’s  website  at 
h  ttp :  /  /  www.  rand .  org/ard .  html . 
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Summary 


In  2003  the  Department  of  Defense  (DoD)  revised  its  acquisition  policy 
to  include  the  Total  Life  Cycle  Systems  Management  (TLCSM)  directive, 
which  calls  for  “cradle-to-grave”  management  of  weapon  and  materiel  systems. 
In  line  with  this  revised  policy,  the  Army  made  a  substantial  organizational 
change,  creating  Life  Cycle  Management  Commands  (LCMCs)  in  2004  to  give 
Army  Materiel  Command  (AMC)  logisticians  more  input  into  acquisition 
processes  and  move  toward  TLCSM.  The  Army  has  also  made  significant 
information  systems  changes,  such  as  fielding  the  Logistics  Information 
Warehouse  (LIW),  to  facilitate  life  cycle  management  of  equipment. 

Still,  recent  reports  have  described  cases  of  critical  life  cycle  management 
(LCM)  decisions  and  supporting  analyses  being  hindered  by  problems  with  life 
cycle  sustainment  (LCS)  data,  i.e.,  information  about  the  operations,  support, 
and/or  disposal  of  Army  equipment.  Additional  steps  may  therefore  be  needed 
to  ensure  that  Army  information  systems  provide  managers  and  analysts  with 
access  to  high-quality,  comprehensive  LCS  data.  Recognizing  this,  the  Deputy 
Chief  of  Staff,  G-4,  Headquarters  Department  of  the  Army  (DA  G-4), 

Resource  Integration  Directorate  sponsored  a  study  to  assess  the  LCS  data 
currently  available  in  Standard  Army  Management  Information  Systems 
(STAMIS). 

Specifically,  we  examined  the  extent  to  which  STAMIS  capture 
information  needed  for  critical  life  cycle  management  decisions  and  analyses. 
Focusing  on  Army  ground  systems,  our  research  approach  included  three 
components:  a  review  of  articles  on  military  and  commercial  LCM  decisions  (to 
identify  types  of  LCS  data  needed);  interviews  with  personnel  who  regularly 
use,  access,  or  manage  Army  LCS  data;  and  the  direct  access,  review,  and 
analysis  of  standard  Army  database  extracts  by  our  research  team. 


Data  Needs  for  Life  Cycle  Management  Decisions 

In  evaluating  Army  LCS  Data,  we  began  by  considering  key  LCM 
decision  areas  and  associated  analyses.  The  decision  areas  considered  included 
acquisition  strategy,  upgrade  planning,  renewal  planning,  scheduled  service 
updates,  maintenance  workforce  planning,  budgeting,  new  system  design,  and 
system  performance.  Based  on  documents  on  commercial  and  military  LCM 
best  practices,  we  then  identified  a  set  of  prescribed  LCS  data  elements  for 
LCM  analyses.  The  set  includes  vehicle  demographics  (information  that 
uniquely  identifies  an  item  or  describes  its  physical  attributes,  owner,  or 
location);  operations  data  (information  about  readiness  and  usage  of  an  item); 
and  maintenance  and  disposal  data  (information  about  maintenance  actions 
and  about  the  removal  of  assets  from  the  fleet).  We  then  investigated  the  degree 
to  which  STAMIS  provides  the  elements  for  the  various  analytic  needs. 

Assessment  of  Life  Cycle  Sustainment  Data 

Our  evaluation  of  STAMIS  LCS  data  was  based  on  criteria  in  three 
categories:  ease  of  access  and  use,  quality,  and  historical  span  of  the  data. 
Overall,  the  findings  suggest  that  Army  data  policies,  processes,  and  systems 
require  substantial  changes  to  support  LCM  analyses  effectively. 

General  Obstacles  to  Access  and  Use  of  LCS  Data 

Statements  from  interviewees,  along  with  the  research  team’s  own 
experiences  with  gathering  LCS  data,  revealed  a  set  of  obstacles  to  data  access; 
for  example,  it  is  not  always  clear  whether  certain  data  exist  or  where  to  find 
them.  While  many  databases  are  centrally  located  in  LIW,  others  are  not  and 
require  a  separate  application  to  obtain  a  system  account.  Second,  access  request 
forms  for  some  systems,  like  the  Operating  and  Support  Management 
Information  System  (OSMIS),  ask  the  applicant  to  specify  those  portions  of  the 
information  system  he/she  will  need  to  access.  The  “catch-22”  is  that,  to  get  an 
account,  one  must  first  specify  the  portions  of  the  information  system  one  will 
use,  but  one  cannot  determine  which  portions  will  be  useful  unless  one  already 
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has  an  account  with  full  access.  Third,  when  applying  for  accounts,  analysts 
who  have  security  clearances,  common  access  cards  (CACs),  and  approval  to 
work  on  Army-sponsored  projects  may  nevertheless  face  long  waits  for 
approval.  In  some  systems  an  account  is  later  frozen  or  revoked  if  a  user  does 
not  log  on  to  the  system  for  30  days. 

Assessment  of  Demographic  Data  on  Army  Equipment 

Our  review  of  demographic  data  on  Army  equipment  indicates  that  such 
data  elements — especially  serial  numbers,  manufacture  dates,  and 
weight/volume  (cube)  measurements — tend  to  be  reasonably  accessible  but  low 
quality.  Serial  number  errors  are  widespread,  and  because  of  such  errors, 
different  databases  often  have  different  versions  of  the  same  serial  number. 
Serial  number  discrepancies  make  it  difficult  to  link  vehicle  data  from  different 
sources  (usage  and  maintenance  data)  by  serial  number.  Also,  manufacture 
dates  in  the  TAMMS  Equipment  Database  (TEDB)  have  inaccuracies.  Many 
do  not  correspond  to  the  serial  number  sequence  or  are  not  plausible,  for 
example,  M2A2  ODS  (Operation  Desert  Storm)  vehicles  that  have 
manufacture  dates  preceding  Operation  Desert  Storm. 

Assessment  of  Data  on  the  Operation  of  Army  Equipment 

Like  equipment  demographic  data  in  STAMIS,  operations  data  have 
considerable  limitations.  In  particular,  data  on  rounds  fired  are  only  available 
for  a  few  end  items  and  require  several  approval  processes  to  access. 
Additionally,  missing  and  implausible  odometer  readings  are  prevalent  in 
Logistics  Integrated  Database  (LIDB)  usage  data.  Units  often  have  many 
months  of  missing  odometer  readings.  When  odometer  readings  are  present, 
errors  are  common,  suggesting  that  vehicles  have  negative  monthly  usage  or 
unreasonably  high  monthly  usage. 

Assessment  of  Data  on  the  Maintenance  and  Disposal  of  Army  Equipment 

Our  review  of  equipment  maintenance  and  disposal  data  in  STAMIS 
suggests  that  such  data  have  moderate  levels  of  accessibility  and  historical  span 
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but  are  low-to-medium  in  quality.  There  is  little  financial  information  about 
disposal  of  Army  equipment.  Also,  there  are  few  records  of  scheduled  field 
maintenance  and  no  records  of  non-deadlining,  unscheduled,  organizational- 
level  field  maintenance.  In  addition,  Army  STAMIS  generally  lack  data  on 
equipment  renewal  (reset,  recapitalization,  or  refurbishment)  by  serial  number. 


Impact  of  Data  Limitations 


Key  LCM  Decisions  Are  Hindered  by  Data-Quality  Issues 


Data  Element 

Acquisition 

strategy: 

Assess 

remaining  life  of 
current  systems 

Upgrade 
planning:  Assess 
failure  rates  & 
profiles  (end  Item 
and  component) 

Renewal 
planning:  Assess 
failure  rates,  costs, 
&  readiness  over 
time 

Scheduled 
service 
updates: 
Assess  failure 
rates  &  profiles 

Serial  Number 

O 

Model 

X 

X 

X 

x 

Q. 

2 

Age  (Manufacture  Date) 

? 

Welght/cube/cost 

X  (cost) 

X  (cost) 

E 

to 

Unit 

X 

Location 

X 

X 

X 

X 

i  g 

Readiness  levels  &  dates 

X 

X 

X 

X 

8.2 
o  is 

Usage  (miles,  hours,  rounds 
fired,  fuel  consumed)  &  dates 

Configuration  mgmt. 

X 

X 

X 

X 

*TO 

O  — 

C  TO 

TO 

Maintenance  type  <sch/ 

unsch)  &  echelon  (field, 
sustainment,  renewal)  &  dates 

c  o 

TO  Q. 

Parts  used  &  cost 

c  £ 

*TO  ® 

Labor  hours  &  cost 

s 

Maintenance  capacity 

Disposal 

RAND  LC  Sustainment  Data  31  01/10 


After  evaluating  LCS  data  elements  available  in  Army  STAMIS,  we  found 
that  most  LCM  analyses  are  not  well  supported  by  available  data.  The  table 
above  shows  data  elements  in  rows,  decisions  and  associated  analyses  in 
columns,  and  an  X  indicating  each  data  element  (row)  needed  for  a  given 
analysis  (column).  The  cells  are  shaded  based  on  a  data  element  quality  rating: 
green  for  high,  yellow  for  medium,  and  red  for  low.  Most  cells  were  shaded  red 
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or  yellow,  an  indication  that  existing  data  are  not  sufficient  for  effective  analyses 
and  quantitatively  well-supported  decisions  in  the  areas  of  acquisition  strategy 
(with  respect  to  replacement),  upgrade  planning,  renewal  planning,  and 
scheduled  service  updates.  A  similar  table  in  the  main  text  (see  page  55)  shows 
that  existing  data  also  do  not  support  decisions  and  analyses  related  to 
maintenance  workforce  planning,  budgeting,  new  system  design,  and  system 
performance  as  well  as  they  should.  Consistent  with  these  tables,  Army 
personnel  interviewed  for  this  study  described  negative  effects  of  LCS  data  gaps 
on  critical  analyses  and  decisions. 

Factors  Contributing  to  Data  Issues 

It  is  likely  that  three  sets  of  factors  contribute  to  the  aforementioned  data 
issues  and  their  effects:  (1)  policy  factors  (limitations  of  existing  Army 
maintenance  policies),  (2)  design  factors  (information  system  design  features), 
and  (3)  execution  factors  (how  Army  doctrine  and  policy  are  carried  out  by 
personnel).  An  example  of  a  policy  factor  is  that  Army  Regulation  (AR)  750-1 
specifies  relying  heavily  on  Program  Managers  (PMs)  to  ensure  that  renewal 
data  are  gathered  and  analyzed  but  does  not  specify  that  STAMIS  should 
archive  such  data  by  serial  number.  A  system  design  factor  is  that  serial 
numbers,  manufacture  dates,  odometer  readings,  and  maintenance  data  are 
input  manually;  consequently,  they  are  subject  to  keystroke  and  logic  errors  as 
well  as  input  reliability  problems.  A  policy  execution  factor  is  the  tendency  for 
deployed  units  to  treat  usage  and  readiness  reporting  as  optional,  and  another  is 
the  lack  of  enforcement  of  reporting  policies.  Also  not  well  executed  are  policies 
concerning  Army  access  to  contractor  maintenance  data. 

Potential  Impact  of  STAMIS  Changes  in  Progress 

Over  the  past  decade,  several  programs  were  initiated  to  change  STAMIS 
structure  and  content:  the  Global  Combat  Support  System-Army  (GCSS-A), 
the  Logistics  Modernization  Program  (LMP),  and,  more  recently,  Item  Unique 
Identification  (IUID).  Subject  matter  experts  indicate  that  all  three  may  address 
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some  of  the  data  issues  discussed  in  this  document,  potentially  providing 
improved  asset  visibility,  faster  run  times  for  reports,  better  database  access,  and 
less  manual  data  entry. 

However,  interviewees  also  raised  concerns  that  suggest  these  initiatives 
should  not  be  seen  as  a  cure-all  for  LCS  data  issues.  Some  cautioned  that  Army 
business  rules  do  not  yet  incorporate  the  broad,  enterprise  perspective  needed 
for  GCSS-A  and  LMP.  While  GCSS-A  promises  to  capture  needed  field-level 
maintenance  data  that  are  currently  missing  or  of  low  quality,  doing  so  will  also 
require  changes  in  maintenance  reporting  practices,  and  GCSS-A  data  quality 
cannot  yet  be  evaluated  in  an  operational  environment.  It  was  reported  that 
LMP  does  not  capture  key  data  elements  that  legacy  systems  have  overlooked — 
renewal  data  by  serial  number,  for  one.  In  the  case  of  IUID,  a  concern  is  the 
potential  for  varying  degrees  of  implementation,  given  that  it  is  an  unfunded 
requirement. 

Recommendations 

Although  the  impact  of  GCSS-A,  LMP,  and  IUID  is  still  uncertain,  it  is 
clear  that  there  are  currently  a  number  of  opportunities  for  improving  the  ease- 
of-access/use,  quality,  and  historical  span  of  Army  LCS  data.  Further  expanding 
LIW  to  centralize  additional  data  and  streamlining  account  approval  processes 
(via  a  list  of  authorized  account  users  for  unclassified  systems)  would  increase 
ease  of  access.  Additional  query  options — such  as  eliminating  restrictions  that 
require  data  to  be  extracted  in  a  piecemeal  fashion — would  increase  ease  of  use 
by  analysts. 

Data  quality  could  be  enhanced  by  greater  error-  and  mistake-proofing  in 
information  systems.  To  reduce  manual  data  entry,  a  one-time  input  of  a 
vehicle’s  serial  number  and  manufacture  date  could  populate  multiple  systems. 
Also,  embedded  automated  data-reporting  instruments  could  be  required  in 
new  Army  vehicles  and  added  to  vehicles  with  onboard  diagnostic  systems  that 
already  capture  needed  data. 
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To  improve  LCS  data  capture  overall,  a  particularly  valuable  step  would 
be  a  methodical  review  and  revision  of  Army  data  policies,  with  analyst  input  in 
the  review  process;  this  step  may  better  align  data  policies  with  strategic 
decision-making  and  analytical  needs.  Army  policies  should  specify  all  data 
elements  that  STAMIS  need  to  capture. 

Additionally,  an  in-depth  examination  of  GCSS-A  and  LMP  should  be 
conducted  against  the  data  needs  identified  in  this  research;  this  could  reveal 
modifications  needed  for  new  systems  to  provide  critical  LCS  data  elements. 
Taking  a  comprehensive  approach  to  LCS  data  improvement  will  help  ensure 
that  managers  and  analysts  have  the  information  needed  to  manage  Army 
equipment  life  cycles  effectively. 
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Glossary 


AEL 

Army  Equipment  Loss 

AEPS 

Army  Electronic  Product  Support 

AMC 

Army  Materiel  Command 

AMSS 

Army  Materiel  Status  System 

AR 

Army  Regulation 

ASA  [ALT] 

Assistant  Secretary  of  the  Army  [Acquisition,  Logistics,  and 
Technology] 

ASV 

Armored  Security  Vehicle 

AWPS 

Army  Workload  Performance  System 

CAC 

Common  Access  Card 

CCSS 

Commodity  Command  Standard  System 

CE/MHE 

Combat  Engineer/Material  Handling  Equipment 

COTS 

Commercial  Off-the-Shelf 

DAU 

Defense  Acquisition  Ekiiversity 

DFPS 

Defense  Forces  and  Public  Security 

DMOPS 

Depot  Maintenance  Operations  Planning  System 

DOL 

Directorate  of  Logistics 

DoD 

Department  of  Defense 

DRMO 

Defense  Reutilization  and  Marketing  Service 

DS 

Direct  Support 

ECR 

Equipment  Control  Record 

ERP 

Enterprise  Resource  Planning 

ESA 

Enterprise  Soldier  Aviation 
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FEDLOG 

Federal  Logistics  Database 

FORSCOM 

Forces  Command 

FSC 

Federal  Supply  Classification 

GAO 

Government  Accountability  Office 

GCSS-A 

Global  Combat  Support  System-Army 

GCV 

Ground  Combat  Vehicle 

GS 

General  Support 

HEMMT 

Fleavy  Expanded  Mobility  Tactical  Truck 

HQDA 

Fleadquarters  Department  of  the  Army 

IFV 

Infantry  Fighting  Vehicle 

IEAP 

Integrated  Logistics  Analysis  Program 

IFSC 

Integrated  Logistics  Support  Center 

IUID 

Item  Unique  Identification 

KPP 

Key  Performance  Parameter 

ECM 

Life  Cycle  Management 

FCMC 

Life  Cycle  Management  Command 

ECS 

Life  Cycle  Sustainment 

LIDB 

Logistics  Integrated  Database 

LIN 

Line  Item  Number 

LIW 

Logistics  Information  Warehouse 

LMP 

Logistics  Modernization  Program 

LOGSA 

Logistics  Support  Activity 

LRIP 

Low-Rate  Initial  Production 

MATDEV 

Materiel  Developer 

MCN 

Management  Control  Number 

MMIS 

Modification  Management  Information  System 
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NUN 

National  Item  Identification  Number 

NMC 

Not  Mission  Capable 

NS  LIN 

Nonstandard  Line  Item  Number 

NSN 

National  Stock  Number 

O&M 

Operations  and  Maintenance 

ODS 

Operation  Desert  Storm 

ORG  WON 

Organizational  Work  Order  Number 

OSD 

Office  of  the  Secretary  of  Defense 

OSMIS 

Operating  and  Support  Management  Information  System 

PBUSE 

Property  Book  Unit  Supply-Enhanced 

PEO 

Program  Executive  Officer 

PM 

Program  Manager 

PMCS 

Preventive  Maintenance  Checks  and  Services 

PRON 

Procurement  Request  Order  Number 

REP 

Request  for  Proposals 

RIDB 

Readiness  Integrated  Database 

RRAD 

Red  River  Army  Depot 

SAFM-CE 

Assistant  Secretary  of  the  Army,  Financial  Management  and 
Comptroller-Cost  and  Economics 

SALE 

Single  Army  Logistics  Enterprise 

SAMS-E 

Standard  Army  Maintenance  System-Enhanced 

SDS 

Standard  Depot  System 

SLAMIS 

SSN-LIN  Automated  Management  and  Integrating  System 

SSN 

Standard  Study  Number 

STAMIS 

Standard  Army  Management  Information  Systems 

TAMMS 

The  Army  Maintenance  Management  System 
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TEDB 

TAMMS  Equipment  Database 

TLCSM 

Total  Life  Cycle  Systems  Management 

TOC 

Total  Operating  Cost 

TPER 

Theater  Provided  Equipment  Refurbishment 

UIC 

Unit  Identifier  Code 

UII 

Unique  Item  Identifier 

ULLS-G 

Unit  Level  Logistics  System-Ground 

1.  Background  and  Research  Approach 
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Life  Cycle  Sustainment  Data  Gaps  in 
Standard  Army  Management  Information  Systems: 
Needs,  Gaps,  and  Opportunities 


Sponsor:  HQDA  G-4 


In  2003  the  Department  of  Defense  (DoD)  revised  its  acquisition  policy 
to  include  the  Total  Life  Cycle  Systems  Management  (TLCSM)  directive, 
which  calls  for  “cradle-to-grave”  management  of  weapon  and  materiel  systems. 
TLCSM  was  specifically  defined  as  “the  implementation,  management,  and 
oversight,  by  the  designated  Program  Manager,  of  all  activities  associated  with 
the  acquisition,  development,  production,  fielding,  sustainment,  and  disposal 
of  a  DOD  weapon  system  across  its  life  cycle”  (Office  of  the  Secretary  of 
Defense,  2003).  A  principal  motivation  for  the  policy  revision  was  to  ensure 
greater  attention  to  system  sustainment — in  particular,  operations  and  support, 
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which  have  been  estimated  to  consume  70  to  80  percent  of  a  weapon’s  total 
ownership  cost  (Andrew  et  ah,  2003;  Redman,  2009). 

In  line  with  the  revised  DoD  policy,  the  Army  made  a  substantial 
organizational  change  to  move  toward  TLCSM.  The  Assistant  Secretary  of  the 
Army  (Acquisition,  Logistics,  and  Technology)  (ASA [ALT))  and  the  Army 
Materiel  Command  (AMC)  launched  the  Army’s  Life  Cycle  Management 
(LCM)  initiative  in  2004  to  “enhance  the  input  of  logisticians  into  acquisition 
processes  pertaining  to  current  and  future  sustainment  and  readiness,  reduce 
costs,  improve  quality,  get  products  to  the  Soldier  faster,  and  implement  a  more 
holistic  approach  to  product  development  and  system  support”  (HQDA  Army 
Posture  Statement,  2008).  Central  to  the  initiative  was  the  creation  of  Life 
Cycle  Management  Commands  (LCMCs),  which  aligned  AMC  Major 
Subordinate  Commands  with  Program  Executive  Officers  (PEOs)  in  the  ALT 
community  to  give  AMC  logisticians  more  input  into  acquisition  processes 
(HQDA,  2004). 

The  Army  has  also  made  significant  information  systems  changes  in 
support  of  TLCSM.  In  2006  the  AMC  Logistics  Support  Activity  (LOGSA) 
fielded  the  Logistics  Information  Warehouse  (LIW),  which  provides  a  common 
point  of  entry  to  the  Logistics  Integrated  Database  (LIDB),  the  Integrated 
Logistics  Analysis  Program  (ILAP),  WebLOG  tools,  and  other  databases. 
LOGSA  continues  to  expand  and  improve  the  content  of  LIW.  Similarly,  the 
Assistant  Secretary  of  the  Army,  Linancial  Management  and  Comptroller-Cost 
and  Economics  (SALM-CE)  and  CALIBRE  Systems  continue  to  add  new 
reports  and  capabilities  to  the  Operating  and  Support  Management 
Information  System  (OSMIS). 

Still,  additional  steps  may  be  needed  to  ensure  that  Army  information 
systems  provide  managers  and  analysts  with  access  to  high-quality, 
comprehensive  life  cycle  sustainment  (LCS)  data — i.e.,  information  about  the 
operations,  support,  and/or  disposal  of  Army  equipment.  Recent  reports  have 
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described  cases  of  crucial  LCM  decisions  and  supporting  analyses  being 
hindered  by  LCS  data  problems.1  For  instance,  the  Army  Audit  Agency 
(2008:16)  noted  that,  due  to  a  lack  of  accurate  data  on  contractor  resets,  “Army 
leadership  and  Congress  could  not  rely  on  the  reported  reset  status  of  wheeled 
and  tracked  vehicles  for  decision-making  purposes.”2  Additionally,  in  a  study  of 
program  management  in  the  U.S.  military,  Garrett  and  Rendon  (2007:394) 
concluded,  “We  currently  are  limited  in  our  understanding  of  life-cycle  costs 
for  our  legacy  systems  due  to  the  lack  of  reliable  information  databases. 

Without  that  knowledge,  we  are  limited  in  estimating  the  impact  of  TOC 
[Total  Operating  Cost]  reduction  efforts  on  those  life-cycle  costs.”  Similarly,  as 
McCoy  (2009:41)  pointed  out, 

A  critical  component  of  effective  life-cycle  systems  management  is 
knowing  how  much  readiness  each  additional  dollar  buys.  In  an  ideal 
situation,  fleet  managers  have  adequate  information  to  analyze  the 
overall  costs  of  sustainment  alternatives  effectively  and  select  an  option 
that  ensures  the  weapon  system  will  provide  the  lowest  total  cost  of 
ownership  consistent  with  an  acceptable  level  of  availability  .  .  . 

However,  the  information  required  to  perform  this  kind  of  analysis  is 
not  available  in  the  current  environment. 

General  Dunwoody  (2009:4)  highlighted  the  value  of  such  data  when  she 
stated  that  “renewing  our  focus  on  sustaining  equipment  through  every  phase 
of  the  life-cycle  process”  calls  for  “a  secure  stream  of  readily  accessible  and 
accountable  information.”  Thus,  it  is  important  to  conduct  a  comprehensive 
review  of  Army  LCS  data,  determining  data  needs,  gaps,  and  opportunities  for 
addressing  those  gaps. 


1  Please  see  Appendix  A  for  a  list  of  reports. 

2  Specifically,  the  accuracy  problem  made  it  difficult  to  determine  the  number  of 
vehicles  available  to  distribute  to  units  and  how  much  to  budget  for  reset  in  the  next 
planning  cycle. 
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The  Director  of  the  Resource  Integration  Directorate  in  the  office  of  the 
Deputy  Chief  of  Staff,  G-4,  Headquarters  Department  of  the  Army  (DA  G-4) 
sponsored  a  study  to  assess  the  LCS  data  currently  available  in  Standard  Army 
Management  Information  Systems  (STAMIS).  Specifically,  we  examined  the 
extent  to  which  STAMIS  capture  information  needed  for  critical  life  cycle 
management  decisions  and  analyses. 
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LCS  Data  Are  Needed  to  Support 
All  Phases  of  Total  Life  Cycle  Management 


RAND 


SOURCE:  Adapted  from  Defense  Acquisition  University  (2009a) 

LC  Sustainment  Data  2  01/10 


The  above  diagram,  which  is  adapted  from  the  DoD  Lifecycle  Framework 
(Defense  Acquisition  University,  2009a)  and  LOGSA  Lifecycle  Logistics  Chart 
(Defense  Acquisition  University,  2009b),  shows  each  phase  of  LCM.  As  the 
dotted  arrows  indicate,  LCS  data  may  serve  as  inputs  to  decisions  and  analyses 
in  the  Pre-Systems  Acquisition,  Acquisition,  and  Sustainment  phases  of  LCM. 

Pre-Systems  Acquisition  includes  Materiel  Solution  Analysis  (assessing 
potential  materiel  solutions  to  a  capability  gap)  and  Technology  Development 
(determining  appropriate  technologies  to  be  integrated  into  the  proposed 
materiel  solution)  (DoD  Instruction  5000.02,  2008:15-16). 

Acquisition  includes  Engineering  and  Manufacturing  Development 
(developing  a  system  and  a  manufacturing  process  for  the  system)  and 
Production  and  Deployment  (achieving  the  desired  operational  capability 
through  low-rate  initial  production  (LRIP)  followed  by  full-rate  production,  if 
the  LRIP  receives  a  favorable  evaluation  (DoD  Instruction  5000.02,  2008:26). 
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Sustainment  includes  Operations  and  Support  (planning  and  managing 
the  operational  readiness  and  maintenance  of  a  system)  and  Disposal  (planning 
and  managing  the  demilitarization  and  removal  of  a  system  from  service  at  the 
end  of  its  useful  life)  (DoD  Instruction  5000. 02). 3 


3  The  Defense  Acquisition  Guide  Lifecycle  Framework  (Defense  Acquisition 
University,  2009a)  treats  disposal  as  part  of  operations  and  support;  however,  the  “LOGSA 
Life  Cycle  Logistics  Chart”  (Defense  Acquisition  University,  2009b)  distinguishes  the  two,  as 
do  other  descriptions  of  life  cycle  management  in  the  private  and  public  sector  (NASA, 
2008).  Here  we  distinguish  the  two  elements  of  sustainment. 
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Project  Focused  on  Assessing  Life  Cycle 
Sustainment  Data  for  Army  Ground  Systems 

Tasks 

1.  Identify  types  and  characteristics  of  Life  Cycle 
Sustainment  (LCS)  data  needed  for  Life  Cycle 
Management  (LCM)  decisions  and  analyses 

2.  Review  sources  of  Army  sustainment  data 

3.  Identify  limitations  in  accessibility,  quality,  and  breadth 
of  data 

4.  Identify  analytic  implications  and  other  consequences 
of  data  limitations 

5.  Identify  factors  affecting  LCS  data  and  opportunities 
for  improvement 

RAND  LC  Sustainment  Data  3  01/10 


Our  assessment  focused  on  LCS  data  for  Army  ground  vehicles  and 
included  five  tasks:  (1)  identifying  a  set  of  key  life  cycle  management  decisions, 
likely  analyses  needed  for  those  decisions,  and  the  data  elements  that,  ideally, 
should  be  available  as  inputs  to  such  analyses;  (2)  examining  existing  LCS  data 
sources  to  determine  the  types  and  characteristics  of  data  captured;  (3) 
identifying  limitations  in  the  available  LCS  data;  (4)  considering  the 
implications  of  those  limitations;  and  (5)  identifying  possible  factors  affecting 
LCS  data  and  opportunities  for  improvement. 
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We  Conducted  Assessments 
Using  Three  Sources  of  Information 

•  Literature  review 

-  Articles  on  military  and  commercial  LCS  data  and 
their  uses 

-  Standard  Army  Management  Information  Systems 
(STAMIS)  briefings  and  manuals 

•  Interviews  with  SMEs  familiar  with  Army  LCS 
data  and  data  sources 

-  TACOM  LCMC;  CECOM  LCMC;  ASA[ALT];  LOGSA; 

HQDA  G-4;  CALIBRE;  PM  Stryker;  RAND  Arroyo 
Center;  CECOM  LCMC;  HQDA  G-4;  ASA(ALT); 

FORSCOM  G-4;  CCAD;  PM  GCSS-A 

•  Hands-on  examination  of  Army  LCS  database 
extracts 

RAND  LC  Sustainment  Data  4  01/10 


Our  research  approach  included  three  components.  The  first  component 
was  a  review  of  articles  on  military  and  commercial  life  cycle  management 
decisions  and  analyses,  focusing  on  the  types  of  LCS  data  they  tend  to 
incorporate.  The  second  component  consisted  of  interviews  with  personnel 
who  regularly  access,  use,  or  manage  Army  LCS  data.  Specifically,  we 
conducted  group  interviews  with  6  managers  and  analysts  in  the  TACOM  Cost 
and  Systems  Analysis  department,  2  managers  at  the  TACOM  ILSC,  5 
managers  at  CECOM’s  Enterprise  Soldier  Aviation  (ESA)  directorate,  5 
managers  at  CECOM’s  Logistics  and  Readiness  Center,  5  managers  at 
ASA  [ALT]  Soldier  and  Maneuver  Systems,  2  managers  at  LOGSA,  and  4 
managers  at  HQDA  G-43.  We  also  conducted  individual  interviews  (1  each) 
with  managers  at  TACOM  ILSC,  CALIBRE,  PM  Stryker,  RAND  Arroyo 
Center,  HQDA  G-4,  LORSCOM  G-4,  Corpus  Christi  Army  Depot,  and  PM 
GCSS-A.  Appendix  B  lists  the  semi-structured  interview  questions. 
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For  the  third  component  of  the  research  approach,  the  research  team 
accessed  standard  Army  information  systems,  documented  our  experiences  in 
accessing  and  retrieving  LCS  data  from  those  sources,  and  then  reviewed  and 
analyzed  the  database  extracts.  Appendix  C  lists  the  information  systems 
discussed  in  this  study. 
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Overarching  Finding:  Army  Data  Policies , 
Processes ,  and  Systems  Are  Not  Well-Suited 

to  LCM  Analyses 

•  LCS  databases  are  not  designed  for  ease  of  access 
and  use  by  Army  analysts 

•  LCS  data  have  substantial  quality  (accuracy  and 
completeness)  problems 

•  Some  LCS  data  important  to  analyses  are  not 
captured  or  archived 

•  Policy,  design,  and  execution  factors  contribute  to 
data  problems 

RAND  LC  Sustainment  Data  5  01/10 


The  above  slide  provides  a  brief  summary  of  key  study  findings,  which  the 
next  section  discusses  in  detail.  Overall,  the  findings  suggest  that  Army  data 
policies,  processes,  and  systems  do  not  effectively  provide  the  LCS  data  needed 
to  support  LCM  analyses.  A  number  of  information  system  features  hinder 
access  to  and  the  use  of  STAMIS  LCS  data  for  analysis  purposes.  Additionally, 
STAMIS  LCS  data  tend  to  have  substantial  quality  problems.  Also,  some  LCS 
data — especially  some  types  of  maintenance  actions — that  are  important  to 
analyses  either  (a)  are  not  captured  at  all  by  STAMIS  or  (b)  are  captured  but 
not  archived.  Reasons  for  these  data  problems  include  limitations  in  Army 
maintenance  data  policies,  current  information  system  designs,  and  practices  of 
units  and  organizations  responsible  for  executing  Army  data  policies. 
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2.  Findings 


We  Considered  Potential  Data  Needs  for  a  Set  of 
Key  LCM  Decisions  and  Supporting  Analyses 


•  Acquisition  strategy 

-  Assess  remaining  life  of  current  systems 

•  Upgrade  planning  (component  or  end  item) 

-  Assess  inherent  failure  rates  and  profiles 

•  Renewal  planning 

-  Assess  failure  rates,  costs,  and  readiness  over  time 

•  Scheduled  service  updates 

-  Assess  failure  rates  and  profiles 

•  Maintenance  workforce  size  and  skill  needs 

-  Calculate  workload  over  time 

•  Budgeting 

-  Calculate  system  O&M  costs 

•  New  system  design  needs 

-  Assess  usage,  maintenance  patterns,  design  issues 
for  current  systems 

•  System  performance  with  respect  to  KPPs 

-  Assess  readiness  and  operational  capability 


Promoting 
reliability  and 
availability 

Resource  / 
budget 
planning 

Materiel 

development 


RAND 
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Data  Needs  for  Life  Cycle  Management  Decisions 

In  evaluating  Army  LCS  data,  we  began  by  considering  LCM  decision 
areas,  analyses  that  support  decisions  in  those  categories,  and  data  elements 
needed  for  those  analyses.  Key  Army  LCM  decision  areas  tend  to  fall  into  one 
or  more  of  three  categories:  (1)  promoting  equipment  reliability  and 
availability,  (2)  determining  budget/resource  needs,  and  (3)  developing 
materiel.  Early  in  a  system’s  life  cycle,  critical  decision  areas  that  fall  in  the  first 
two  categories  include  acquisition  strategy  development,  upgrade  planning,  and 
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renewal  planning.4  An  acquisition  strategy  refers  to  a  framework  for  planning, 
directing,  contracting,  and  managing  a  system  to  achieve  program  objectives 
within  the  resource  constraints  imposed  (Defense  Acquisition  University, 
2009c).  An  example  of  an  LCM  analysis  that  may  inform  acquisition  strategy 
decisions  is  an  assessment  of  the  remaining  life  of  current  systems;  systems  with 
little  remaining  life  are  likely  to  call  for  replacement  or  recapitalization  earlier 
than  other  systems. 

Just  as  acquisition  strategies  are  needed  for  new  systems,  upgrade  plans  are 
needed  for  end  item  or  components  remaining  in  use.  Upgrade  planning  may 
include  decisions  about  which  systems/components  need  modernization,  how 
many  will  be  upgraded,  and  the  upgrade  and  fielding  schedules.  Assessments  of 
failure  rates  and  profiles  can  inform  such  plans.  As  suggested  by  Dumond, 
Eden,  Mclver,  and  Shulman  (1994:28),  data  collection  and  analyses  of 
performance  and  failure  patterns  in  newly  fielded  as  well  as  mature  systems  can 
help  detect  and  isolate  design  deficiencies  that  call  for  engineering  changes. 

Another  decision  area  that  has  become  particularly  prominent  during 
Operation  Iraqi  Freedom  (OIF)  and  Operation  Enduring  Freedom  (OEF)  is 
renewal  planning,  currently  one  of  the  Army’s  top  priorities  (GEN  Chiarelli, 
2009).  Here  we  use  the  term  renewal  to  refer  to  reset,  refurbishment,  overhaul, 
upgrade,  and  recapitalization  processes  that  restore  equipment  damaged  during 
combat.  (Each  of  these  renewal  processes  has  a  different  scope  of  work  and  level 
of  funding;  recap  has  the  most  extensive  scope,  followed  by  upgrade  and 


4  Because  acquisition,  upgrade,  and  renewal  efforts  may  be  intended,  in  part,  to 
increase  equipment  reliability  and  availability,  and  because  such  decisions  affect  (and  are 
affected  by)  budgets,  they  fall  in  the  categories  of  promoting  reliability/ availability  and 
resource/budget  planning.  Consistent  with  this  categorization,  the  Defense  Acquisition 
Guidebook  (Defense  Acquisition  University,  2009a)  notes  that  life  cycle  sustainment 
planning  (to  maintain  reliability)  and  resource  management  are  both  considerations  in  the 
selection  of  an  acquisition  strategy. 
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overhaul,  then  reset  and  refurbishment  [Boucher,  2007]. 5)  Renewal  decisions 
may  concern  the  type  of  renewal  needed  for  a  system,  when  and  where  such 
processes  should  occur,  the  portion  of  the  fleet  to  be  renewed,  and  how  much 
to  budget  for  renewal.  Analyses  of  the  impact  of  age,  prior  renewal, 

OPTEMPO,  location,  and  other  predictors  on  failure  rates,  sustainment  costs, 
and  readiness  may  inform  such  decisions. 

A  fourth  LCM  decision  area,  which  is  in  the  category  of  promoting 
reliability  and  availability,  is  scheduled  service  updates — that  is,  identifying 
necessary  changes  in  preventive  maintenance  checks  and  services  (PMCS).  The 
prescribed  set  of  services  in  the  Army’s  quarterly,  semi-annual,  and  annual 
PMCS  schedules  need  to  be  updated  as  vehicles  are  modified  or  used  under 
different  conditions.  A  deployed  field  maintenance  company  recently  reported 
that  performing  annual  services  twice  as  often  (semi-annually)  was  more 
efficient  and  effective  in  Iraq  than  performing  the  prescribed  quarterly  and 
semi-annual  services,  given  the  high  OPTEMPOs  and  extreme  temperatures 
(King,  2009).  Failure  rates  and  profiles  need  to  be  analyzed  systematically  to 
assess  PMCS  effectiveness  and  determine  when  such  changes  are  warranted  on  a 
large  scale.  As  LaFergola  (2010)  observed  in  his  article  on  commercial  lift  truck 
fleet  management,  data  on  maintenance  performed  helps  “facility  managers 
recognize  circumstances  in  which  unexpected  maintenance  was  required  that 
may  go  beyond  the  [current  scheduled  maintenance  program].”6 


5  Recap  includes  rebuild  (returning  items  to  “like-new” — zero  mile — condition)  plus 
upgrade  (enhanced  capabilities);  overhaul  is  strictly  rebuild  (return  to  like-new  condition) 
without  upgrade;  and  reset  returns  equipment  to  its  predeployment  “10/20  standard” 
condition  (HQDA  G-4,  DALO-MNN,  2008:26;  Boucher,  2007).  In  theater  this  return  to 
10/20  condition  is  known  as  refurbishment — more  specifically,  Theater  Provided 
Equipment  Refurbishment. 

6  PMCS  revision  may  be  appropriate  when  analysts  find  a  trend  such  as  the  higher  rate 
of  M2  Bradley  transmission  failures  that  occurred  in  Iraq.  Boyd  (2005:21)  recalled  that  M2 
Bradley  transmission  failures  frequently  occurred  in  Iraq  due  to  extra  weight  (about  5,000 
pounds  of  reactive  armor)  and  higher  mileage.  As  he  observed,  “Failure  trends  such  as  this 
highlight  the  importance  of  PMCS  and  proper  scheduled  maintenance.” 
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A  key  LCM  decision  area  in  the  category  of  resource/budget  planning 
involves  maintenance  workforce  planning — that  is,  determining  who  will 
maintain  equipment  (organic  versus  contract  personnel)  and  how  many 
maintainers  are  needed.  As  a  recent  U.S.  Government  Accountability  Office 
(GAO)  study  of  depot  maintenance  planning  pointed  out,  planning  for  future 
maintenance  requirements  calls  for  “timely  and  reliable  information  from  their 
major  commands  on  both  the  amounts  and  types  of  workloads  they  should 
expect  to  receive  in  future  years”  (Solis,  2009:4). 

Budget  preparation  is  also  an  integral  part  of  life  cycle  planning.  To 
support  budget  requests  to  Congress  and  facilitate  budget  execution 
(management  of  awarded  funds),  annual  operations  and  maintenance  (O&M) 
budget  forecasts  are  needed,  as  well  as  estimates  of  O&M  costs  during  the  life 
of  a  weapon  system.  Such  analyses  may  include  comparing  the  costs  and 
benefits  of  contractor  versus  organic  logistics  support  (Warren,  2002a). 
Additionally,  to  estimate  the  cost  of  a  new  system,  analysts  “begin  with  the 
costs  of  earlier  systems  of  a  similar  nature,  that  is,  analogous  systems.  The 
‘cleaner’  the  cost  database  of  the  predecessor  system(s),  the  higher  [will  be]  the 
confidence  in  cost  estimates  for  the  new  system”  (Garrett  and  Rendon, 
2007:202). 

A  critical  LCM  decision  area  related  to  materiel  development  is  new 
system  design  needs  (assessing  how  well  existing  systems  are  functioning  under 
current  operating  conditions).  Repair  data  for  current  equipment  may  reveal 
design  flaws  or  vulnerabilities  that  need  to  be  addressed  in  new  system  designs. 

Another  materiel  development  decision  entails  determining  whether  a 
system  under  development  has  sufficient  performance  with  respect  to  key 
performance  parameters  (KPPs)  to  proceed  with  fielding.  KPPs  are  attributes  or 
characteristics  of  a  system  considered  most  essential  for  an  effective  military 
capability  (Naegle,  2005).  When  threshold  or  objective  KPPs  for  a  system  are 
not  met,  the  acquisition  program  for  that  system  may  be  reevaluated  or 
terminated. 
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Table  2.1  includes  examples  of  key  decisions,  and  associated  LCS  data 
needs,  that  interviewees  in  this  study  mentioned. 

Table  2.1 

Interviewee  Statements:  Decisions  Requiring  LCS  Data 


Interviewee 

Statement 

ASA[ALT] 

manager 

"Engines  might  be  the  number  one  cost  driver  on  a  particular  vehicle, 
and  the  manufacturer  may  not  be  desiring  to  continue  to  produce 
that  engine  because  it's  inefficient  or  uneconomical  to  produce,  or 
the  technology  may  be  old  .  .  .  There's  logistics  information  that 
supports  the  argument  for  investing  in  upgrades." 

TACOM  Cost  and 
Systems  Analysis 
manager 

"At  different  stages  of  the  life  of  a  weapon  system,  there's  a  decision 
about  whether  to  continue  on  or  stop  spending  money.  At  those 
stages,  you  have  a  decision  maker  at  the  OSD  level.  Department  of 
the  Army  level,  or  at  the  command  level  at  TACOM,  depending  on 
the  dollar  value  of  the  program.  The  decision  maker  wants  to  know 
'How  much  is  it  going  to  cost?  What  capability  is  it  going  to  provide 
me?'  We  use  sustainment  data  in  developing  these  life  cycle  cost 
estimates." 

"We  also  use  sustainment  data  in  providing  decision  makers  with 
alternative  courses  of  action.  These  may  include  looking  at 
alternatives  for  reducing  the  cost  of  sustainment — buying  parts  for  a 
system  .  .  .  The  economic  analysis  looks  at  the  costs  associated  with 
either  increasing  the  reliability  of  a  part  or  reducing  the  cost  of  a 
part  and  compares  the  reduction  in  the  actual  operating  costs." 

CECOM  ESA 
manager 

"When  the  contractor  is  developing  a  weapon  system  for  us,  they'll 
base  estimates  on  analogy — i.e.,  'it  was  this  much  for  a  similar 
system.'  They  [use]  good  engineering  estimates,  but  they  are  not 
collected  from  the  field.  It  would  be  nice  to  get  data  back  from  the 
field." 

Drawing  from  documents  on  commercial  and  military  LCM  best 
practices  (such  as  General  Administration  of  State  of  Washington,  2006;  Office 
of  the  Secretary  of  Defense,  Cost  Analysis  Improvement  Group,  2007;  Sowerby 
and  Lauria,  2007;  Wyrick  and  Storhaug,  2003),  the  research  team  identified  a 
set  of  prescribed  LCS  data  elements  for  LCM  analyses.  The  set  includes: 

•  Vehicle  demographics — information  that  uniquely  identifies  an  item 
or  describes  its  physical  attributes,  owner,  or  location. 
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•  Operations  data — information  readiness  and  usage  of  an  item. 

•  Maintenance  and  disposal  data — information  about  maintenance 
actions  and  about  the  removal  of  assets  from  the  fleet. 

For  each  of  the  focal  decision  areas,  our  research  team  considered  which 
data  elements  contribute  to  rigorous  supporting  analyses,  including  well- 
specified  statistical  models. 
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LCM  Decisions  Require  Data  Elements  in  the  Categories  of  System 
Demographics,  Operations,  Maintenance,  and  Disposal 


Data  Element 

Acquisition 

strategy: 

Assess 

remaining  life  of 
current  systems 

Upgrade 
planning:  Assess 
failure  rates  & 
profiles  (end  item 
and  component) 

Renewal 
planning: 
Assess  failure 
rates,  costs,  & 
readiness  over  time 

Scheduled 
service 
updates: 
Assess  failure 
rates  &  profiles 

Serial  Number 

X 

X 

X 

X 

3 

IE 

Model 

X 

X 

X 

X 

Q. 

(0 

Age  (Manufacture  Date) 

X 

X 

X 

X 

O) 

o 

Weight/cube/cost 

X  (cost) 

X  (cost) 

E 

0) 

Unit 

X 

Location 

X 

X 

X 

X 

&  c 

Readiness  levels  &  dates 

X 

X 

X 

X 

a.  2 

O  co 

Usage  (miles,  hours,  rounds 
fired,  fuel  consumed)  &  dates 

X 

X 

X 

X 

Configuration  mgmt. 

X 

X 

X 

X 

*0) 

o  — 

C  03 
to  </> 

Maintenance  type  (sch/ 

unsch)  &  echelon  (field, 
sustainment,  renewal)  &  dates 

X 

X 

X 

X 

£  & 

Parts  used  &  cost 

X 

X 

X 

X 

C  S 

'<5  Q 

Labor  hours  &  cost 

X 

X 

X 

X 

s 

Maintenance  capacity 

X 

Disposal 

X 
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The  table  in  this  slide  has  data  elements  listed  in  rows;  decisions  and 
supporting  analyses  listed  in  columns;  and  an  X  indicating  each  data  element 
(row)  needed  for  a  given  analysis  (column). 

While  not  preferred,  it  is  certainly  possible  to  do  such  analyses  with  fewer 
data  elements  than  those  marked  with  an  X.  Indeed,  prior  studies  of  Army 
fleets  have  often  proceeded  without  serial-level  data — i.e.,  datasets  that  tie  age, 
usage,  location,  and  other  variables  to  individual  vehicle  serial  numbers; 
instead,  they  used  data  at  a  higher  level  of  analysis  (Chappell  et  ah,  2007; 
Congressional  Budget  Office,  2007).  However,  the  use  of  aggregate  data  in 
such  studies  can  obscure  patterns  that  would  otherwise  be  detected  with 
individual-level  data.  Consistent  with  this  point,  in  a  study  of  Canadian  Air 
Force  (CAF)  equipment  performance,  Chouinard  (1997:40)  noted  that 
aggregate  data  in  the  CAF’s  Aircraft  Maintenance  Management  Information 
System  (AMMIS)  were  not  ideal  for  reliability  studies: 
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It  is  important  to  note  that  equipment  failure  data  collected  under 
the  AMMIS  do  not  provide  all  the  necessary  details  for  traditional 
failure  data  analysis.  In  particular,  the  data  collected  for  any 
particular  type  of  equipment  are  aggregated  by  month  across  the 
entire  fleet  of  aircraft  on  which  the  equipment  is  installed.  These 
aggregate  data  are  important  for  fleet  management,  but  they  obscure 
information  about  individual  failure  events  which  could  be  used  for 
failure  analysis. 

Thus,  it  is  important  for  LCS  data  elements  to  enable  rigorous  study 
methodologies  with  appropriate  levels  of  analysis. 

As  the  above  table  shows,  decisions  and  analyses  related  to  reliability  and 
availability  tend  to  call  for  most  of  the  demographic,  operations,  and 
maintenance/disposal  data  elements.  As  mentioned  earlier,  renewal  planning 
may  be  informed  by  an  assessment  of  the  impact  of  age,  prior  renewal, 
OPTEMPO,  location,  and  other  predictors  on  failure  rates,  sustainment  costs, 
and  readiness.  Such  analyses  ideally  include  longitudinal  demographic, 
operations,  and  maintenance/disposal  data  at  the  individual  vehicle  level.  For 
each  serial  number,  analysts  need  the  corresponding  year  of  manufacture,  the 
equipment  model,  usage,  and  renewal  dates  and  types  to  determine  predictor 
variables  in  statistical  models.  For  the  same  serial  numbers,  analysts  need 
readiness  and  maintenance  records  to  develop  the  statistical  models.  Such  data 
elements  and  models  may  reveal  which  types  of  renewal  tend  to  yield  the  most 
readiness  and  cost  benefits,  the  systems  for  which  effects  are  greatest,  and  the 
optimal  timing  of  renewal. 
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LCM  Decisions  Require  Data  Elements  in  the  Categories  of  System 
Demographics,  Operations,  Maintenance,  and  Disposal 


Maintenance 
workforce  size 
&  skill  needs: 
Calculate 
workload 

Budgeting: 
Calculate  system 
08>M  costs 

New  system 
design  needs: 
Assess  usage, 
maint  patterns  for 
current  systems 

System 
performance: 
Assess 
readiness  & 
oper  capability 

Serial  Number 

X 

X 

X 

X 

o 

Z 

Model 

X 

X 

X 

X 

a 

2 

Age  (Manufacture  Date) 

X 

O) 

o 

Weight/cube/cost 

X  (cost) 

X 

E 

3 

Unit 

X 

Location 

X 

X 

Readiness  levels  &  dates 

X 

X 

X 

Ope 

atior 

Usage  (miles,  hours,  rounds 
fired,  fuel  consumed)  &  dates 

X 

X 

X 

Configuration  mgmt. 

X 

X 

X 

■8 

o  _ 

s  s 

Maintenance  type  (sch /  unsch) 

&  echelon  (field,  sustainment. 

renewal)  &  dates 

X 

X 

X 

c  o 

<D  CL 

Parts  used  &  cost 

X 

X 

1  s 

Labor  hours  &  cost 

X 

X 

X 

2 

Maintenance  capacity 

X 

Disposal 
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Decisions  and  analyses  related  to  resource/budget  planning  and  materiel 
development  also  call  for  many  of  the  data  elements  listed.  In  order  to  identify 
critical  requirements  for  new  systems,  analyses  may  consider  how  and  where 
vehicles  are  currently  being  used,  which  systems  are  having  readiness  problems 
and/or  incurring  high  maintenance  costs  because  of  such  usage,  and  the 
subsystems  that  are  particularly  susceptible  to  failure  under  those  conditions. 
Thus,  readiness,  usage,  location,  configuration,  and  maintenance  data  on 
current  systems  are  among  the  likely  inputs  to  analyses  identifying  new  system 
design  needs. 

Also,  to  justify  O&M  budget  decisions,  it  may  be  necessary  to  develop 
models  that  estimate  maintenance  costs  for  items  based  on  age,  usage,  and 
configuration.  Additionally,  readiness  data  may  contribute  to  an  assessment  of 
the  benefits  of  maintenance  expenditures,  further  informing  O&M  budget 
decisions. 
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We  Identified  Criteria  for  Evaluating  LCS  Data/Databases 
and  Organized  Them  into  Three  Categories 


Category 

Assessment  Criteria 

Ease  of 
Access  and 
Use 

•  Are  the  data  available  on  a  STAMIS? 

•  Is  gaining  approval  for  access  burdensome? 

•  Is  the  database  well-documented  and  user-friendly? 

•  Can  the  data  be  readily  linked  to  other  data? 

Quality 

•  Are  the  data 

-  accurate? 

-  complete  (e.g.,  portion  of  fleet  covered)? 
precise  (e.g.,  serial  number  level)? 

-  up-to-date? 

Historical  span 

•  How  many  years  of  data  are  available? 

Sources  of  types  &  categories:  Jagadish  et  al.  (2007) 

http://www.eecs.umich.edu/db/usable/usability.pdf 
Department  of  Transportation  (2000) 

RAND  http://www.itsdocs.fhwa.dot.gov/JPODOCS/REPT_MIS/13580.html 


LC  Sustainment  Data  9  01/10 


Assessment  of  Life  Cycle  Sustainment  Data 

The  previous  section  described  a  set  of  critical  LCM  decision  areas  and 
outlined  the  data  elements  needed  for  analyses  in  those  areas.  In  this  section  we 
present  an  assessment  of  the  extent  to  which  STAMIS  provide  those  data. 

Our  evaluation  of  STAMIS  LCS  data  was  based  on  criteria  in  three 
categories:  ease  of  access  and  use,  quality,  and  historical  span  of  the  data.  This 
slide  shows  the  criteria  corresponding  to  each  category. 

Similar  criteria  have  been  used  in  prior  studies  evaluating  data  and  data 
sources.  Jagadish  and  colleagues  (2007)  noted  that  the  usability  of  a  database, 
“the  ability  to  get  information  into  and  out  of  the  system  easily  and  efficiently,” 
is  as  important  as  the  capability  of  the  database.  They  suggested  that  usability 
includes  such  factors  as  the  degree  to  which  users  have  to  “stitch  information 
together  to  answer  most  of  the  real  queries,”  whether  “a  system  hinders  users 
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from  querying  the  data  in  the  way  they  want,”  and  the  difficulty  of  locating  a 
particular  piece  of  data. 

Others  have  focused  on  data  quality  and  access  as  evaluation  criteria.  The 
Intelligent  Transportation  Society  (2000)  emphasized  that  its  Advanced 
Traveler  Information  Systems  need  to  have  high  data  quality  (interpreted  as  the 
degree  of  accuracy,  timeliness,  breadth/depth  of  coverage,  and  detail)  as  well  as 
sufficient  access  (availability  to  others).  The  organization’s  guidelines  point  out 
that  either  “good  data  with  limited  or  no  access  or  inadequate  data  with 
sufficient  access”  can  be  problematic  (ITS,  2000). 

Another  consideration  is  the  amount  of  historical  data  retained.  Referring 
to  studies  about  the  relationship  between  age  and  human  behavior,  Singer  and 
Willett  (2003:9)  noted  that 

.  .  .  too  many  empirical  researchers  seem  willing  to  leap  from  cross- 
sectional  data  that  describe  differences  among  individuals  of 
different  ages  to  making  generalizations  about  change  over  time. 

They  explained  that  the  problem  with  such  an  approach  is  that  individuals  in 
different  age  groups  belong  to  different  cohorts  that  experienced  different 
school  curricula  and  life  events: 

In  statistical  terms,  cross-sectional  studies  confound  age  and  cohort 
effects  (and  age  and  history  effects)  and  are  prone  to  selection  bias 
(Singer  and  Willett,  2003:10). 

By  the  same  token,  cross-sectional  data  are  not  ideal  for  studying  vehicle 
performance  over  time.  Comparing  the  performance  of  a  5 -year-old  vehicle  and 
a  separate  1 0-year-old  vehicle  is  not  equivalent  to  comparing  the  performance 
of  a  single  vehicle  at  age  5  and  age  10.  In  general,  longitudinal  data  are 
preferred  for  assessing  vehicle  changes  over  time. 
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Interviewees  and  the  Research  Team  Encountered 
Obstacles  to  the  Access  of  LCS  Data 

•  Not  all  data  accessible  via  one  portal  (i.e.,  LIW) 

•  Account  request  forms  (e.g.,  for  LIW  and  OSMIS)  ask  which  parts 
of  the  information  system  one  will  need  to  access 

-  a  “catch-22”:  difficult  to  know  without  already  having  an 
account 

•  Lengthy  waits  for  approval  in  some  cases  and  refusals  in  others 
despite  clearance,  CAC  card,  and  need  for  access 

•  Even  with  an  approved  system  account,  some  applications  not 
accessible  without  a  special  approval  from  other  organization 

•  Need  to  enter  some  databases  every  30-60  days  to  retain  access 
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General  Obstacles  to  Access  and  Use  of  LCS  Data 

Statements  from  interviewees  and  the  research  team’s  own  experiences 
with  gathering  LCS  data  revealed  a  set  of  obstacles  to  data  access.  First,  it  is  not 
always  clear  to  analysts  whether  certain  data  exist  or  where  to  find  them.  While 
many  databases  are  centrally  located  at  the  Logistics  Information  Warehouse, 
others  are  not.  Army  Electronic  Product  Support  (AEPS),  Army  Workload 
Performance  System  (AWPS),  Modification  Management  Information  System 
(MMIS),  and  Operating  and  Support  Management  Information  System 
(OSMIS)  are  not  part  of  LIW,  and  each  requires  a  separate  application  process 
for  a  system  account. 

In  addition,  access  request  forms  for  some  systems  (AWPS,  OSMIS,  and 
LIW)  ask  the  applicant  to  specify  those  portions  of  the  information  system 
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he/she  will  need  to  access.7  However,  it  is  difficult  to  know  all  that  is  available 
on  an  information  system — and  which  portions  will  be  relevant  and  useful  to 
one’s  analyses — without  having  an  account  and  access  to  all  portions  of  the 
information  system.  A  description  from  a  manual,  if  available,  is  often  not 
sufficient  to  determine  the  level  of  analysis  of  the  data,  the  locations  and 
equipment  covered,  and  the  data  fields  included  in  each  portion  of  an 
information  system.  The  “catch-22”  is  that,  to  get  an  account,  one  must  specify 
the  portions  of  the  information  system  one  will  use,  but  one  cannot  determine 
what  one  will  use  unless  one  has  an  account. 

In  some  cases,  analysts  are  granted  access  to  an  information  system  only  to 
find  that  some  of  the  modules  in  that  information  system  are  not  accessible 
without  further  approval  processes.  In  the  AWPS  website,  for  instance,  separate 
applications  were  needed  to  access  the  Depot  Maintenance  Operations 
Planning  System  (DMOPS)  and  Red  River  Army  Depot  (RRAD)  AWPS,  two 
of  multiple  databases  listed  in  AWPS.  Also,  even  if  one  obtains  an  account  in 
AEPS,  one  will  not  automatically  have  access  to  the  gun  card  firing  data. 
Instead,  the  site  states,  “You  do  not  have  Authorized  Access  granted  for  this 
application.  To  obtain  access,  please  contact  one  of  the  Application  POCs 
[people  on  the  ‘AMC  Gun  Card  Points  of  Contact’  list].”  Similarly,  LIW  has  a 
link  to  the  SSN-LIN  Automated  Management  and  Integrating  System 
(SLAMIS),  but  one  still  needs  a  valid  logon  account  with  SLAMIS  to  use  it. 

Once  access  to  an  information  system  is  granted,  it  can  easily  be  lost. 
According  to  the  LOGSA  help  desk,  an  LIW  account  will  be  frozen  if  it  is  not 
used  for  30  days  and  deleted  if  it  is  not  used  for  90  days.  An  interviewee  in  the 
TACOM  Cost  and  Systems  Analysis  department  noted,  “The  databases  we  use 
have  varying  password  requirements  that  at  times  make  it  difficult  to  retain 


7  The  System  Access  Request  web  page  for  LIW  states  that,  for  the  broader  “controlled 
access,”  “you  will  be  given  basic  access  and  provided  a  list  of  controlled  applications  (reports 
to  select  from  that  are  available  according  to  your  credentials).  Your  authorizing  official  will 
have  to  approve  access  to  all  groupings  based  on  what  is  needed  to  do  your  duties.” 
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access — requirements  such  as  expiration  dates  [if  the  account  is  not  used]  for 
30-60  days.  A  project  may  require  use  of  a  particular  database  for  a  short  time; 
then  its  use  may  not  be  required  for  another  six  months.  A  person  should  not 
have  to  keep  entering  the  database  on  a  monthly  basis  just  to  retain 
membership  for  when  it  is  needed.” 
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Interviewees  and  the  Research  Team  Encountered 
Obstacles  to  the  Use  of  LCS  Data 

•  Delays  and  disruptions — particularly  with  large  data  pulls 

-  Lengthy  run  times 

-  Frequent  “time  outs”  (need  to  log  in  multiple  times) 

•  Documentation  and  user  interface  challenges 

-  Databases  with  overlapping  functions 

-  Missing  labels  (miles  vs.  kilometers)  in  LIDB 

-  No  detailed  OSMIS  manual — just  high-level  slide  presentations 

-  ILAP  work  orders  still  listed  as  GS-DOL  and  DS  in  Shop  Operations 
module,  even  though  maintenance  is  now  field  and  sustainment 

•  Flexibility  issues 

-  Cannot  select  region  (e.g.,  Iraq,  Afghanistan)  for  consolidated 
property  listing  from  PBUSE 

-  Piecemeal  data  pulls 

>  Usage  by  serial  number  pulled  one  month  at  a  time  in  OSMIS, 
one  year  at  a  time  in  LIDB 

•  In  ILAP  Shop  Operations  module,  can  pull  completed  work 
RANin  orders  for  a  NIIN,  but  not  for  a  list  of  N  INs  or  an  FSC  „  . 
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Just  as  there  are  obstacles  to  accessing  information  systems  with  LCS  data, 
there  are  impediments  to  the  use  of  those  data.  In  LIW  and  Property  Book 
Unit  Supply-Enhanced  (PBUSE),  the  run  time  for  a  report  can  range  from 
minutes  to  hours,  depending  on  the  amount  of  data  requested  and  the  number 
of  people  using  the  system.  Said  a  FORSCOM  manager,  “LIW,  because  of  the 
way  it  operates,  is  time-consuming.  It  is  lengthy  in  running  the  scrubs  I  want.” 

In  addition,  users  often  need  to  type  on  their  keyboard  or  select  menu 
items  regularly  to  ensure  that  they  are  not  logged  off  of  a  system  due  to 
inactivity.  The  AEPS  website  has  a  30-minute  timer  that  counts  down  when  a 
user  is  not  typing  or  selecting  menu  items.  If  the  counter  reaches  zero,  the 
session  times  out.  As  a  second  FORSCOM  manager  noted,  “There’s  a  time 
limit  where  you  log  on  and  the  clock  starts  ticking.  You  can’t  just  minimize  it; 
you  can’t  continue  to  do  your  work  without  requesting  additional  time.  It’s 
minor,  but  it’s  a  nuisance.”  Similarly,  ILAP  times  out  after  a  few  minutes 


-26- 


without  activity.  Since  downloading  a  large  data  file  from  ILAP  can  easily  take 
30  minutes  or  longer,  the  downloading  process  will  stop — and  the  link  to  ILAP 
will  close — unless  the  user  presses  a  key  every  few  minutes  to  show  activity. 

User  interface  features  for  some  systems  can  make  it  difficult  to  determine 
which  application  to  use.  Some  of  the  applications  have  overlapping  functions, 
so  it  is  not  always  clear  which  application  is  most  appropriate  for  an  analyst’s 
purposes.  Within  LIW,  work  orders  can  be  obtained  from  ILAP  or  from  the 
SAMS  Research-Work  Order  Detail  report;  Army  asset  on-hand  quantities  can 
be  obtained  from  multiple  LIDB  reports  (Army  Assets,  Total  Property  Book 
Assets)  as  well  as  PBUSE;  serial  number  lists  are  available  from  LIDB  Ground 
Equipment  Tracker  and  Army  Serial  Number  Tracker. 

Other  user  interface  issues  hinder  interpretation  of  LCS  data.  First,  while 
LIW  has  detailed  online  documentation,  OSMIS  does  not;  only  high-level  slide 
presentations  explaining  OSMIS  can  be  found  at  the  website.  Second,  the 
LIDB  Serial  Number  Usage  Report  does  not  indicate  whether  the  odometer 
readings  are  in  miles  or  kilometers.  Third,  the  ILAP  completed  work  order 
reports  are  called  “DS  Work  Orders”  and  “GS-DOL  Work  Orders,”  labels  that 
correspond  to  the  Army’s  previous  four-level  maintenance  structure, 8  even 
though  the  Army  currently  has  a  two-level  (field  and  sustainment)  maintenance 
structure,  with  field  maintenance  divided  into  organizational  and  support-level 
work  orders.  Because  the  work  order  report  titles  do  not  correspond  to  the 
Army’s  current  maintenance  structure,  it  may  not  be  readily  apparent  to  ILAP 
users  what  portion  of  all  maintenance  actions  the  reports  are  capturing. 

Information  system  flexibility  issues  were  also  reported.  In  the  PBUSE 
Consolidated  Property  Listing  (which  provides  a  listing  of  all  assets  required, 
authorized,  on  hand,  and  due  in),  one  can  limit  the  data  pull  to  a  specific  Unit 


8  The  four-level  maintenance  structure  included  organization-level,  direct  support 
(DS),  general  support  (GS),  and  depot.  The  Directorate  of  Logistics  (DOL)  has  traditionally 
been  a  provider  of  maintenance  above  the  DS  level  (McCurry,  1995).  The  transition  from 
four-level  to  two-level  maintenance  is  discussed  in  more  detail  later  in  this  report. 
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Identifier  Code  (UIC)  or  a  range  of  Line  Item  Numbers  (LINs),  but  one 
cannot  restrict  the  output  to  a  geographic  region,  command,  or  division.  While 
one  report  is  running,  a  second  one  cannot  be  requested.  If  the  user  requests  a 
large  report  by  mistake,  the  request  cannot  be  cancelled,  and  it  may  not  be 
possible  to  submit  another  request  for  hours. 

In  other  information  systems,  some  data  can  only  be  obtained  in  a 
piecemeal  fashion.  In  OSMIS,  the  Monthly  Management  Report  (MMR) 
provides  monthly  usage  and  odometer  readings  from  Unit  Level  Logistics 
System-Ground  (ULLS-G)  by  serial  number,  but  users  can  only  request  one 
month  at  a  time  when  running  the  report.  Users  can  request  up  to  12  months 
of  data  when  running  the  LIW/LIDB  Serial  Number  Usage  Report.  If  users 
need  5  to  1 0  years  of  odometer  readings  for  a  longitudinal  analysis,  the  report 
needs  to  be  run  5  to  10  times.  In  the  ILAP  Maintenance  (Shop  Operations) 
module,  one  can  request  closed  DS  and  GS-DOL  work  orders  for  a  given 
National  Item  Identification  Number  (NIIN)  but  not  for  a  list  of  NIINs.  If  one 
is  familiar  with  the  SQL  programming  language,  it  is  possible  to  use  the 
Dynamic  Query  module  to  request  closed  DS  work  orders — but  not  GS-DOL 
work  orders — for  a  specific  list  of  NIINs. 

In  summary,  there  are  noteworthy  impediments  to  the  access  and  use  of 
STAMIS  LCS  data.  Data  are  located  in  multiple  information  systems  with 
separate  application  processes,  and  such  application  processes  can  be  lengthy, 
with  legitimate  applicants  sometimes  receiving  partial  access  or  being  refused 
access.  When  granted  access,  database  users  tend  to  encounter  delays  and 
disruptions  when  running  reports,  ambiguous  user  interfaces  and 
documentation,  and  database  flexibility  issues. 
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Demographic  Data  Tend  to  Be  Poor-to-Medium  Quality 


Data  Element 

Data 

Source 

Ease  of  Use/ 
Access 

Quality 

History 

ILAP 

H 

L 

Serial  Number 

LIDB 

H 

L 

— 

PBUSE 

M 

L 

NSN,  LIN,  NSLIN 

LIDB 

PBUSE 

H  (M  for  NSLIN) 

M 

M-H 

(L  for  NSLIN) 

— 

Age  (Year  of  Manufacture) 

LIDB  / 
TAMMS 

H 

L-M 

— 

Weight/Cube/Cost 

FEDLOG 

H 

L/L/H 

L 

Unit 

LIDB 

H 

M 

M 

PBUSE 

M 

M 

L 

Location 

LIDB 

PBUSE 

H 

L 

M 

M 

M 

L 

•Rating  based  on  access/use  for  analysis  purposes 


H  =  High  M  =  Medium  L  =  Low  —  NA 
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Assessment  of  Demographic  Data  on  Army  Equipment 

The  preceding  section  discussed  general  limitations  to  the  access  and  use 
of  LCS  data  in  Army  information  systems.  In  this  section  we  present  findings 
from  an  assessment  of  the  access/ use,  quality,  and  historical  span  of 
demographic  LCS  data  elements. 

In  the  above  table,  the  first  column  contains  the  demographic  data 
element  that  was  rated;  the  second  column  contains  sources  of  each  element; 
and  the  third,  fourth,  and  fifth  columns  contain  the  research  team’s  ratings 
corresponding  to  each  element  and  data  source.9 


9  Appendix  D  contains  a  table  that  explains  the  basis  for  each  medium  (M)  and  low 
(L)  rating. 
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As  indicated  by  the  ratings,  most  demographic  data  on  Army  equipment 
are  reasonably  accessible  after  one  has  an  account  in  LIW,  and  except  in  the 
case  of  theater-provided  equipment,  LIDB  provides  historical  unit  and  location 
data  for  more  than  a  decade.  However,  demographic  data — especially  serial 
numbers,  manufacture  dates,  and  weight/volume  (cube)  measurements,  tend  to 
be  low  quality.  Subsequent  slides  illustrate  quality  issues  that  correspond  to 
some  of  the  L  ratings  in  the  table. 
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Fleet  Counts  Vary  Not  Only  by  Database,  but  also 
by  Report  Within  Same  Database 


Example:  #  of  Armored  Security  Vehicles  (ASVs) 
Textron  Weekly  Email  Update  to  Item  Mgr 

TAMMS  Equipment  Database* 

PM  Assets  Issued  to  Units* 

LIDB  Total  Unit  Assets* 

LIDB  PB  Assets,  queried  by  NIIN* 

PBUSE  (Direct  Pull)  I 

LIDB  Army  Assets  by  MACOM* 

LIDB  PB  Assets,  queried  by  LIN* 


54  Battle 
Losses 

^^MO2094 

^^^■1844 

U]l560 
■  i486 
1 1486 
1 1 461 
1444 
1429 


ILAP  026  data 


1099 


Serial  Number  Usage  Data*  I  1924 


RAND 


NOTE:  Data  pulled  09/17/09  *  Standard  LIW  queries 

for  property  accountability 

study  by  Lewis  et  al.  (2009).  Not  total  fleet  counts 
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Fleet  Counts  and  Serial  Numbers 

A  recent  study  of  Army  property  accountability  (Lewis  et  al.,  2009)  found 
that  different  Army  information  systems — and  different  reports  within  the 
same  information  system — yielded  different  on-hand  counts  for  the  same  type 
of  vehicle,  even  though  the  data  were  gathered  on  the  same  day.  On  September 
17,  2009,  the  PBUSE  Consolidated  Property  Listing  showed  that  there  were 
1,460  Armored  Security  Vehicles  (ASVs),  while  LIDB  Property  Book  Assets 
showed  that  there  were  1,486.  The  TAMMS  Equipment  Database,  which  (like 
LIDB  Property  Book  Assets)  is  accessed  via  LIW,  indicated  that  there  were 
1 ,844  ASVs.  All  these  counts  were  hundreds  of  vehicles  short  of  the  ASV 
delivered  fleet  size  of  2,094  reported  by  Textron,  the  manufacturer.  Equipment 
on-hand  counts  in  STAMIS  are  a  reflection  of  the  serial  numbers  provided  by 
units.  Serial  numbers  that  are  either  not  reported  or  not  reported  accurately 
contribute  to  inconsistent  fleet  counts. 
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A  Variety  of  Serial  Number  Errors  Are  Common  in  STAMIS, 
Particularly  Maintenance  Systems 


Error  Description 

Example 

Source 

User-created  serial 

“SEE  LIST” 

ILAP  work  order  for  Ml  A1 

number 

“N/A” 

“HOME  STATION” 

Registration  number 

NPOPD2 

ILAP  work  order  for  PLS  M1075 

listed  instead  of 
serial  number 

NPOPCU 

Registration  +  full  or 

3ADR3309R/MV033 

ILAP  work  order  for  M3A2  ODS 

partial  serial  number 

MV046Y-3277R 

Dashes  or  other 

34949-6608732 

PBUSE  data  on  Fork  Lift  Trk  (T48944) 

symbols 

31/Q2113 

PBUSE  data  on  M931A2  Trk  TRAC 

3ADR3258R*2WO'S 

ILAP  data  on  M3A2  CFV 

Shortened  serial 

IFUJATCGI^fMTOT^) 

1  FUJATCG24jPM~10l5i?3 

LIDB  serial  number  usage  report  for 

numbers 

M915A3 

*172  serial  numbers  beginning  with 
1FUJATCG24  disappear  from  the 
reports  in  Aug  2007  but  have  recent 
manufacture  dates  &  usage  up  to 
that  point.  A  data  cleanup  error? 

RAND  LC  Sustainment  Data  14  01/10 


Serial  number  errors  include  entering  an  alternative  to  the  serial  number 
such  as  “N/A”  or  the  equipment  registration  number;  combining  the 
registration  number  and  serial  number;  inserting  dashes  or  other  characters  in 
the  serial  number;  and  shortening  the  serial  number.  These  types  of  errors  are 
found  in  extracts  from  LIDB  and  PBUSE  but  are  especially  common  in 
maintenance  data. 
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Serial  Number  Discrepancies  Make  It  Difficult  to  Link  Different 
Types  of  Data  Corresponding  to  the  Same  Vehicle 

Example  of  Serial  Number  Cleanup  Needed  for  FLU  419  (LIN  T34437): 

We  initially  found  6,679  distinct  serial  numbers  associated  with  LIN  T34437 
in  PBUSE,  RIDB,  TEDB,  Usage,  and  MMIS  databases.  After  a  preliminary 
“cleanup,"  there  were  2,082  distinct  serial  numbers. 


Below:  Although  identical  6-digit  sequences  appear  within  the  serial 
numbers  from  each  database,  none  of  the  original  serial  numbers  can  be 
matched  without  cleaning. 


“Clean” 

Usage 

MMIS 

RIDB* 

PBUSE 

TEDB 

Serial 

Serial 

Serial 

Serial 

Serial 

Serial 

450072 

HM450072 

OHM450072 

1FG9999HM450072 

1 FG999990HM450072 

90HM450072 

450148 

97HM450148 

7HM450148 

HM450148 

450148 

1 FG999997HM450148 

450423 

93HM450423 

1 FG993HM450423 

450423 

HM450423 

1FG999993HM450423 

451469 

AM451469 

KM451469 

451469 

IFG999990KM451469 

90KM451469 

*Some  of  the  discrepancies  are  due  to  different  databases  having 
different  serial  number  length  restrictions:  e.g.,  maintenance  database 
cuts  off  length  at  15  digits. 
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Because  of  such  errors,  different  databases  often  have  different  versions  of  the 
same  serial  number.  This  slide  shows  four  examples  of  serial  numbers  that  have 
different  forms  in  the  LIDB  Serial  Number  Usage  Reports,  MMIS,  Readiness 
Integrated  Database  (RIDB),  PBUSE,  and  TAMMS  Equipment  Database  (TEDB). 
Some  analyses  call  for  linking  manufacture  dates,  odometer  readings,  work  orders, 
and  other  data  by  serial  number;  however,  serial  number  discrepancies  can  hinder 
this  process.  However,  automated  linking  cannot  be  done  by  analysts  around  the 
Army  as  analytic  needs  develop.  Analysts  typically  need  to  “clean”  the  serial  number 
data  before  linking  serial-level  data  from  different  sources. 

In  this  example,  the  total  number  of  distinct  FLU419  (LIN  T34437  - 
Small  Emplacement  Excavator)  serial  numbers  found  in  different  sources 
appeared  to  be  6,679  before  the  serial  numbers  were  cleaned.  After  cleanup, 
there  was  substantially  greater  overlap  in  the  serial  number  lists  from  different 
sources;  the  number  of  distinct  serial  numbers  went  down  to  2,082. 
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Serial  Numbers  Listings  Vary  Substantially  Across  Databases 


Armored  Security  Vehicle  (ASV) 


TEDB 

(1844) 


RAND 


PBUSE 

(1461) 


PM 

Assets 
(1560) 


M1075  &  M931  have  similar 
patterns,  with  <25%  of  serial 
numbers  appearing 
in  all  3  databases 


Vehicle 

#  serial 
numbers 

In  all  3 
databases 

#  serial 
numbers 
in  2 

databases 

#  serial 
numbers 

In  1 

database 

ASV 

604 

889 

1,275 

M1075 

1,219 

1,808 

4,263 

M931 

2,106 

3,521 

6,177 

NOTE:  Data  pulled  09/17/09  for  property  accountability  study  by 
Lewis  et  al.  (2009) 
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This  Venn  diagram,  also  from  Lewis  et  al.  (2009),  further  illustrates  the 
impact  of  serial  number  discrepancies  across  data  sources.  Of  the  total  set  of 
ASV  serial  numbers  listed  in  TEDB  (N=  1,844),  the  LIDB  PM  Assets  database 
(N=l,560),  and  PBUSE  (N=l,46l)  in  September  2009,  less  than  25  percent 
(N=604  vehicles)  appeared  in  all  three  databases. 

The  table  to  the  right  of  the  Venn  diagram  shows  that  the  overlap  in  serial 
number  lists  from  different  databases  was  not  only  low  for  the  ASV,  but  also  for 
the  PLS  (palletized  load  system)  Ml 075  and  M931  truck.  For  the  Ml 075, 
4,263  serial  numbers  appeared  in  only  one  of  the  three  databases  (i.e.,  TEDB 
only,  PBUSE  only,  or  LIDB  PM  Assets  only);  1,808  appeared  in  only  two  of 
the  three  databases  (i.e.,  TEDB  and  PBUSE,  TEDB  and  PM  Assets,  or  PBUSE 
and  PM  Assets);  1,219  serial  numbers  (only  17  percent  of  the  total  number) 
appeared  in  all  three  databases. 
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Only  Serial  Numbers  Appearing  in  TEDB  Have  Year-of-Manufacture  (YOM) 
Data,  and  Many  of  the  Dates  are  Questionable 

•  Serial  numbers  not  in  TEDB  lack  YOM  data  (e.g.,  44%  of  ASV  serial  numbers 
and  56%  of  PLS  M1075  serial  numbers  in  PBUSE  not  in  TEDB) 

•  Most  serial  numbers  that  are  in  TEDB  have  manufacture  dates,  but  many 
dates  do  not  correspond  to  the  serial  number  sequence  or  variant 

Examples: 


Ml A2  Abrams  Tank 


ser#Cl><o. 

REG  NO. 

YR  iyrfJcTN, 

L/56272251O1, 

JZ07NL 

/  2006 

|A,27227m\ 

JZ07RG 

/  1995 

Za27230M  \ 

JZ07RU 

/  1969 

LA27231M  \ 

JZ07RW 

/  1969 

LA27240M 

JZ07NM 

2007 

LA27246M 

JZ07NZ 

2006 

LA27247M 

JZ07NW 

2006 

LA27252M 

JZ07P1 

2006 

LA27255M 

JZ07QR 

2006 

LA27256M 

JZ07NT 

2006 

LA27257M 

JZ07NN 

2006 

LA27258M  | 

JZ07P3 

\  2006 

\l A27259M  / 

JZ07NK 

\  2006 

\a.28202M  / 

JZ07M2 

\  1995 

1^2 8204 M/ 

JZ07LY 

\  1995 

 LAXS205&4  fc. 

JZ07NH 

A  2006, 

M2A2  ODS 


Dates  precede  Operation  Desert  Storm 


LC  Sustainment  Data  17  01/10 


Dates  do  not  correspond  to  serial  number  sequence 


Because  TEDB  is  the  only  standard  Army  database  containing  equipment 
manufacture  dates  by  serial  number,  analysts  generally  lack  age  data  for  serial 
numbers  that  do  not  appear  in  TEDB.  Of  the  1,461  ASV  serial  numbers  that 
appear  in  PBUSE,  824  are  in  TEDB;  this  means  that  at  least  637  (or  44 
percent)  of  those  serial  numbers  lack  age  data  in  a  central  source. 

About  96  percent  of  the  serial  numbers  that  do  appear  in  TEDB  have 
manufacture  dates  listed,10  but  the  accuracy  of  those  dates  tends  to  be  low. 
Despite  serial  numbers  typically  generated  in  manufacturing  sequence  order, 
many  manufacture  dates  do  not  correspond  to  the  serial  number  sequence; 


10  This  96  percent  figure  is  based  on  data  pulled  from  ILAP  table 
TEDB_SN_DETAILS  for  FSC  2350  (tracked  combat,  assault,  and  tactical  vehicles)  on 
March  30,  2010. 
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serial  numbers  that  are  very  close  in  sequence  (L27227M  and  L27230M)  can 
have  very  different  manufacture  dates  listed  (1995  vs.  1969).  Moreover,  some 
manufacture  dates  listed  for  a  variant  are  well  before  the  date  that  the  variant 
was  first  released — M2A2  ODS  (Operation  Desert  Storm)  vehicles  with 
manufacture  dates  preceding  Operation  Desert  Storm. 
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For  Recapitalized  Vehicles,  Year  of  Manufacture  (YOM)  Sometimes  Reflects 

the  Recap  Date  and  Sometimes  Reflects  Original  Manufacture  Date 

HEMTT  LHS  Vehicles  that  have  been  through  the  Recapitalization,  Repair, 
and  Return  program  (designated  by  model  ending  in  Riy~\ 

In  OEM  data 

MODEL 

SN 

REGNO 

OWNING  UIC 

/yrmpg 

provided  by  PM  HTV, 
recap  dates  for 

Mil  20  range  from 

M1120A2R1 

2  K1J20E 1022490 

NP1WQ5 

WARLAO 

1983  \ 

M112QA2R1 

2  K1J20F 1025861 

NP1UR9 

WACKAO  / 

2002  \ 

M1120A2R1 

2  K1J20G 1020001 

NP1UA1 

WAYHAA  / 

2002  \ 

2000  to  2009. 

M1120A2R1 

2K1J22S1051193 

NP1WQ3 

WARLAO 

1983 

Year  of  manufacture 
i  listed  in  TEDB  is  in 

M1120A2R1 

2K1J23G 1028657 

NP1WQ7 

WARLGO 

2007 

M1120A2R1 

2  K1J23M 1042103 

NP1URB 

WACKFL 

2002 

M1120A2R1 

2K1J24D 1020515 

NP1UA3 

WAYHAA 

2002 

the  2002-2007  time 

M1120A2R1 

2K 1J  25G 1027736 

NP1WQ2 

WARLAO 

1983 

frame  for  some  of 

M1120A2R1 

2  K1J25J 1033494 

NP1WQG 

WARLAO 

1983 

the  recapitalized 

M1120A2R1 

2KU25J10B5990 

NP1WQU 

WG81AA 

1988 

Mil  20  vehicles  and 

M1120A2R1 

2K1J26H 1031599 

NP1WQC 

WARLAO 

1983 

in  the  1980s  and 

M1120A2R1 

2  K1J26K 1036521 

NP1WQE 

WARLAO 

1983 

1990s  for  others. 

M1120A2R1 

2K1J27K 1039640 

NP1WQD 

WARLAO 

1983 

M1120A2R1 

2K1J2XD1021197 

NP1WQ4 

WARLAO 

1983 

M1120A2R1 

2KU2XJ 1033037 

NP1WQ6 

WARLAO 

1983 

M1120A2R1 

33011 

NP1WBG 

WCBGAO 

2002 

M1120A2R1 

333044 

NP1WBK 

WCBGAO 

2002 

Also,  some  dates  do 

M1120A2R1 

333060 

NP1WBH 

WCBGAO  \ 

2002 

not  correspond  to 
original  manufacture 
date  or  recap  date. 

M1120A2R1 

34023 

NP1WPA 

WJ  MSTO  \ 

2002  1 

M1120A2R1 

34103 

NPOEPE 

W2MK2B  \ 

1995  / 

M1120A2R1 

34331 

NPOERZ 

W2MKZB 

1995  / 

M1120A2R1 

34419 

NP1WBJ 

WCBGAO 

V°2  / 

M1120A2R1 

34904 

NP0ET6 

W2MKZB 

19515  y 

K  AIN  U 

TEDB  data  as  of  22  Jan  2010 

LC  Sustainment  Data  18  01/10 

When  vehicles  have  been  recapitalized,  TEDB  sometimes  lists  the 
recapitalization  (recap)  date  instead  of  the  original  year  of  manufacture.  The 
HEMTT  LHS11  vehicles  shown  above  were  all  recapitalized  over  the  past 
decade;  TEDB  lists  the  recap  date  for  some  and  the  original  manufacture  date 
for  others.  This  inconsistent  treatment  of  “YR  MFG”  makes  it  difficult  to  use 
the  data  to  analyze  the  effects  of  age  on  equipment. 


11  Heavy  Expanded  Mobility  Tactical  Truck  Load  Handling  System. 
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A  Nonstandard  Line  Item  Number  (NSLIN)  May  Not  Be  Precise  Enough  to 
Distinguish  High-  and  Low-Priced  Items  in  the  SLAMIS  Catalog 

•  If  the  MCN  or  NSN  for  a  nonstandard  item  is  not  listed  in  a  property 
book  or  other  data  source,  the  NSLIN  will  not  be  sufficient  to 
determine  a  ballpark  price 

NSLIN 

NSLIN  Nomenclature 

Model 

Unit  Price 

FA207Q 

MOTOROLA  2  WAY  RADIO 

MTX  800 

$  500 

FA207Q 

MOTOROLA  2  WAY  RADIO 

L04QKH9PW9AN 

$  6,500 

FA207Q 

MOTOROLA  2  WAY  RADIO 

COMMAND  PLUS  L1680A 

$  19,673 

NSLIN 

NSLIN  Nomenclature 

Model 

Unit  Price 

RB7509 

FIRING  DEVICE 

MRFD-3R  GENERAL  DYNAM 

$  6,800 

RB7509 

FIRING  DEVICE 

ED  FIRESET  SYSTEM 

SPECTRA  PHYSICS  LASER 

$  65,000 

NSLIN 

NSLIN  Nomenclature 

Model 

Unit  Price 

YF2088 

TRUCK  TRACTOR 

CHEVROLET  5700SFI 

$  9,079 

YF2088 

TRUCK  TRACTOR 

GU813  MACK 

$250,000 

RAND 

LC  Sustainment  Data  19  01/10 

Nonstandard  Line  Item  Numbers 

Another  demographic  data  element  with  a  data  quality  issue  is  the 
Nonstandard  Line  Item  Number  (NSLIN).  Traditionally,  items  in  the  federal 
supply  system  have  been  assigned  a  national  stock  number  (NSN)  and  a  line 
item  number  (LIN)  for  cataloging,  requisitioning,  and  accountability.  In  some 
cases,  however,  the  Army  procures  and  uses  items  that  do  not  yet  have  LINs, 
such  as  commercial  off-the-shelf  (COTS)  equipment  or  items  that  are  fielded 
before  item  managers  type-classify  them  and  enter  them  into  the  standard 
catalog  (Dickson,  2006).  Such  items  are  assigned  NSLINs  instead  of  standard 
LINs,  and  they  may  have  management  control  numbers  (MCNs)  instead  of 
NSNs.  While  a  standard  LIN  typically  corresponds  to  a  small  set  of  similar 
items,  an  NSLIN  can  correspond  to  a  wide  variety  of  items,  and  this  can 
present  property  accountability  and  equipment  valuation  problems.  A 
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FORSCOM  manager  described  several  challenges  related  to  NSLINs  and 
SLAMIS  as  follows: 

[SLAMIS]  was  designed  to  allow  us  to  identify  and  track  all  these 
NSLINs,  but  the  problem  is  that  you  and  I  can  go  in  and  request 
LIN  assignments  for  particular  equipment.  You  might  get  a  LIN 
assignment,  and  I  might  get  [a  different]  LIN  assignment  for  the 
same  item.  The  database  might  treat  it  as  2  separate  items.  You  can 
call  [the  equipment]  ABC,  and  I  can  call  it  ABC  1/2.  The  system 
may  not  realize  it’s  the  same  item.  [As  a  result],  the  same  system  can 
have  many  different  NSLINs. 

Also,  you  will  see  a  ton  of  equipment  classified  under  the  same 
NSLIN.  If  I  have  LIN  FA2008,  and  if  I  go  without  knowing  the 
stock  number,  I  can’t  tell  you  what’s  out  there.  I  need  the  stock 
number  [NSN]  or  MCN.  It’s  not  clear  why  there  are  so  many 
different  items  under  the  same  NSLIN — but  all  having  different 
NSNs.  Those  are  some  of  the  nuances  that  make  it  challenging. 

The  above  slide  contains  examples  of  items  with  very  different  prices 
falling  under  the  same  NSLIN.  If  one  wants  to  determine  the  fair  market  value 
of  an  item  so  as  to  determine  the  value  of  a  loss  (as  prescribed  by  AR  735-5 
(HQDA,  2005a),  the  NSLIN  often  will  not  be  sufficient  (Dickson,  2006). 
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In  SLAMIS,  Many  NSLINs  Lack  Sufficient  Precision  for 
Tracking  and  Pricing  Equipment 


•  In  the  SLAMIS  Catalog,  the  same  NSLIN  often  corresponds  to  very 


RAND 


* based  on  SLAMIS  data  downloaded  December  2009 


Average  number 
of  prices  listed 
per  NSLIN:  8 

Average 
difference 
between  max 
and  min  price 
listed:  $114,558 

Average  of  all 
NSLIN  prices: 
$78,491 


LC  Sustainment  Data  20  01/10 


The  cases  on  the  previous  slide  are  not  isolated  examples;  the  above  chart 
suggests  that  many  NSLINs  in  SLAMIS  correspond  to  items  with  very  different 
prices.  The  average  NSLIN  price  was  $78,491  as  of  December  2009,  and  the 
average  difference  between  the  maximum  and  minimum  price  corresponding  to 
the  same  NSLIN  was  $1 14,558.  That  is,  the  average  range  of  prices 
corresponding  to  an  NSLIN  exceeds  the  average  price  of  an  NSLIN. 

For  the  9,251  NSLINs  (out  of  16,206)  with  at  least  2  distinct  prices,  this 
bar  chart  shows  the  percentage  of  NSLINs  that  had  narrow  versus  wide 
differences  between  their  maximum  and  minimum  prices.  Together,  the  3  bars 
on  the  left  indicate  that  about  40  percent  of  the  NSLINs  had  wide  differences 
(ranging  from  $10,000  to  more  than  $500,000). 
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Types  and  Quality  of  Usage  Data  in  STAMIS 

Are  Limited 


Data 

Ease  of  Use/ 

Quality 

History 

Data  Element  (by  serial  no.) 

Source 

Access 

Readiness 

RIDB 

H 

M 

H 

LIDB 

H 

L 

H 

Mileage 

OSMIS 

M 

L 

M 

0) 

O) 

(0 

Operating  Hours 

— 

None 

— 

— 

(/) 

Z> 

Fuel  Consumption 

OSMIS 

H 

L 

— 

Rounds  Fired 

AEPS 

L 

L 

? 

H  =  High  M  =  Medium  L  =  Low  —  NA 
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Assessment  of  Data  on  the  Operation  of  Army  Equipment 

Like  equipment  demographic  data  in  STAMIS,  operations  data  have 
considerable  limitations.  In  particular,  data  on  rounds  fired  have  access 
challenges,  and  data  on  miles  driven  have  substantial  quality  issues.  Ground 
systems’  operating  hours  (including  idling  time)  are  not  captured.  Appendix  D 
lists  the  bases  for  the  M  and  L  ratings  in  the  above  table,  and  subsequent  slides 
in  this  section  illustrate  key  readiness  and  usage  quality  issues. 
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Missing  Data  Reduce  the  Quality  of  Readiness  Reports 


•  RIDB  feedback  reporting  statistics  for  2009  show,  on  average,  monthly 
readiness  reports  were  submitted  by  69%  of  FORSCOM  units  required  to 
report  (749  out  of  1089  units),  with  standard  deviation  of  2% 

•  RIDB  has  2008 — but  not  2009 — data  (as  of  Feb  2010)  for  some  systems 
listed  in  MMDF  “B  Tables”  of  Readiness  Reportable  Items — e.g., 

-  M91 5  series  truck  tractors  (LIN  T61 1 03) 

-  130G  Grader  (LIN  G74783) 

-  Deuce  Tractor  (LIN  T76541) 


RIDB  has  readiness  data  for  a  relatively  small  portion  of  some  fleets 


AE2S  Expert  System  History  provided  on-hand  quantities  of  M1097P1  series  from  Jan  08  to  Nov  09 
RAND  LC  Sustainment  Data  22  01/10 


Readiness  Data 

Equipment  readiness  (percent  fully  mission  capable)  data  are  available  in 
the  LIW  Readiness  Integrated  Database  (RIDB).  However,  missing  data  tend 
to  be  a  problem.  Based  on  a  RIDB  Feedback  Reporting  Statistics  report  for 
2009,  about  a  third  of  FORSCOM  units  that  are  required  to  report  readiness 
are  not  doing  so.  Those  units  that  do  report  readiness  do  not  necessarily  report 
it  for  all  of  their  vehicles.  Consequently,  readiness  data  only  correspond  to  a 
small  percentage  of  some  fleets.  A  TACOM  IFSC  manager  noted,  “My 
[monthly)  report  says  that  there  are  34,995  M2  machine  guns  in  active  use,  and 
the  readiness  report  [says]  9,995  in  active  use.  So  the  readiness  report  is  only 
picking  up  one-third  of  what’s  out  there — again,  it’s  because  units  aren’t 
reporting.”  Consistent  with  his  point,  the  bar  chart  here  shows  that  RIDB  had 
readiness  data  for  about  one-fifth  of  the  HMMWV  M1097P1  fleet  during 
2009. 
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Usage  Data  Quality  Problems  Include  Missing 
and  Implausible  Odometer  Readings 

suggests 

ATLAS  -  LIN  T7 3347  implausible  increase  (decimal  point  error)  negative 

y  /  usage 


7 

10KA0290 

2917 

2917 

3323 

N3614 

37023 

^37190 

37269 

10KA0291 

misi 

>ing 

55204 

27602 

(^27602 

321^ 

)  321 

10KA0392 

5330 

865 

535 

467 

999 

1029 

2180 

In  monthly  LIDB  usage  data  from  Nov  2004  to  Nov  2009, 

On  average,  each  ATLAS  serial  number  had  2.45  months  of  negative  usage 

65  percent  of  the  serial  numbers  (1,342  out  of  2,050)  had  at  least  one  month 
of  negative  usage 

Same  problem  seen  in  usage  data  for  other  systems,  e.g., 

Ml  Abrams,  M2  Bradley,  M1151  HMMWV,  M915  line  haul 

RAND  LC  Sustainment  Data  23  01/10 


Equipment  Usage  Data 

Both  missing  and  implausible  odometer  readings  are  prevalent  in  LIDB 
usage  data.  The  Serial  Number  Usage  Report  provides  monthly  odometer 
readings  by  serial  number  from  as  early  as  1996  (for  those  systems  existing  at 
that  time)  until  the  present.  However,  some  odometer  readings  falsely  indicate 
that  vehicles  have  negative  monthly  usage  (i.e.,  reading  for  month  n+1  less  than 
reading  for  month  n)  or  unreasonably  high  monthly  usage.  Additionally, 
missing  odometer  readings  are  common. 
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A  Unit  May  Not  Report  Vehicle  Usage 
(Odometer  Readings)  for  Many  Months 


UNIT  NO  /NAME 

UIC 

FSC 

NUN 

SERIAL 

REG  NQ. 

YR  MF 

8-Nqu 

S-DtrC 

9-Jan 

9-Fefc 

9-Mc* 

H.-Aar 

TJuU 

0002  ■  IN  BN 

A  CO 

WJTJAO 

2350 

14059886 

2ADR0CC6 

MV06FB 

1995 

0002  •  IN  BN 

A  CO 

WJTJAO 

2350 

14059886 

2ADR0163 

MV06KY 

1996 

1697 

2579 

3055 

3892 

0002 -IN  BN 

A  CO 

WJTJAO 

2350 

14059886 

2ADR0201 

MV06M2 

1996 

0002  -  IN  BN 

A  CO 

WJTJAO 

2350 

14059886 

2ADR3591 

MV02EC 

1987 

1045 

1045 

2127 

2944 

0002 -IN  BN 

A  CO 

WJTJAO 

2350 

14059886 

2ADR3788 

MV06EX 

2009 

0002 -IN  BN 

A  CO 

WJTJAO 

2350 

14059886 

2ADR3989 

MV0ZWG 

1994 

0002 -IN  BN 

A  CO 

WJTJAO 

2350 

14059586 

2ADR4C07 

MV04DJ 

0002 -IN  BN 

A  CO 

WJTJAO 

2350 

14059886 

2ADR4021 

MV01BJ 

2009 

0002 -IN  BN 

A  CO 

WJTJAO 

2350 

14059886 

2ADR4022 

MV02VE 

0002 -IN  BN 

A  CO 

WJTJAO 

2350 

14059886 

2ADR4030 

MV01C4 

0002 -IN  BN 

2ND  A  CO  LBE 

WJTJAL 

2350 

14059886 

2ADR0163, 

MV06KY 

1996 

1222 

0002  -  IN  BN 

2ND  A  CO  LBE 

WJTJAL 

2350 

14059886 

2ADR3163 

MV02R8 

1987 

8865 

8865 

8865 

8865 

8865 

8865 

8865 

0002  -IN  BN 

2ND  A  CO  LBE 

WJTJAL 

2350 

14059886 

2ADR3591 

MV02EC 

1987 

522 

0002 -IN  BN 

2ND  A  CO  LBE 

WJTJAL 

2350 

14059886 

2ADR3646 

MV02W1 

1987 

268 

268 

268 

268 

396 

0002 -IN  BN 

2ND  IN  B  CO  LBE 

WJTJBL 

2350 

12487619 

2ADR3109 

MV020V 

1986 

319 

0002  -  IN  BN 

2ND  IN  B  CO  LBE 

WJTJBL 

2350 

12487619 

2ADR3354, 

MV02RU 

1987 

593 

593 

593 

593 

593 

593 

593 

0002 -IN  BN 

2ND  IN  B  CO  LBE 

WJTJBL 

2350 

14059886 

2ADR3812 

MV06EB 

1987 

0002  -  IN  BN 

CCO 

WJTJC0 

2350 

14059886 

2ADR0101 

MV06J4 

1996 

745 

745 

750 

750 

0002 • IN  BN 

CCO 

WJTJC0 

2350 

14059886 

2ADR3110 

MV0204 

1986 

86 

224 

364 

585 

Excerpt  from  LIDB  Serial  Number  Usage  Report  for  M2A2  ODS 
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Particularly  when  they  are  deployed,  units  often  do  not  report  odometer 
readings  for  many  months.  In  this  example  the  deployed  equipment  items  in  an 
infantry  company  (UIC  “WJTJAO”)  have  almost  no  M2A2  ODS  odometer 
readings  between  November  2008  and  June  2009.  Left-behind  equipment 
(LBE)  items  in  the  rear  detachment  (with  UIC  “WJTJAL”)  have  more  readings, 
although  there  are  still  gaps.12 


12  The  monthly  odometer  readings  for  the  rear  detachment  in  this  example  generally 
do  not  change  from  month  to  month.  One  reason  may  be  that  the  left-behind  equipment  for 
that  unit  was  not  used. 
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Implausible  Usage  Data  Include  Identical  Monthly  Odometer 
Readings  for  Multiple  Vehicles  in  Different  Units 


UIC 

FSC 

NUN 

SERIAL  N< 

REG  NO. 

YRMFG 

/-Nov 

k  6-Dec 

7-Jan 

7-Feb 

Alar 

\  7-Apr 

WAJVBO 

2350 

12487619 

2AD20450 

MV03WV 

1990 

/18302 

\  7396 

7396 

7473 

A1340 

\  7476 

WA4FA0 

2350 

12487619 

2AD20475 

MV03XL 

1991 

/  18302 

\  6338 

6353 

6354 

/  41340 

Y6440 

WAJVTO 

2350 

12487619 

2AD20712 

MV044K 

1990 

/  18302 

\  4249 

4489 

4362 

/  41340 

1374 

WA4FT0 

2350 

12487619 

2AD20968 

MV04K8 

1990 

/  18302 

l  9450 

9474 

9490 

/  41340 

$190 

WAJVTO 

2350 

124875'? 

2AD21090 

MV04NU 

1991 

/  18302 

\  8632 

8633 

8633 

/  41340 

8 

33 

WA4FA0 

2350 

12487619 

2AD21130 

MV04Q0 

1991 

1  18302 

17292 

7294 

7295|  41340 

7 

22 

WAJVBO 

2350 

12487619 

2AD21174 

MV04RA 

1991 

18302 

4896 

4896 

4896 

41340 

41 

13 

WAJVAO 

2350 

12487619 

2AD21264 

MV04V5 

1991 

18302 

15656 

5656 

5705 

41340 

57 

8 

WA4FE0 

2350 

12487619 

2AD21265 

MV04V6 

1991 

18302 

1143 

4489 

1246 

41340 

12 

6 

WAJVAO 

2350 

12487619 

2AD21272 

MV04VD 

1991 

18302 

3961 

4489 

4012 

41340 

40 

3 

WAJVBO 

2350 

12487619 

2AD21274 

MV04VF 

1991 

18302 

3532 

4489 

3601 

41340 

36 

4 

WJL4T0 

2350 

12487619 

2AD21275 

MV04VG 

1991 

18302 

612 

4489 

621 

41340 

6 

6 

WA4FA0 

2350 

12487619 

2AD21276 

MV04VH 

1991 

18302 

9380 

9385 

938! 

41340 

96 

6 

WA4FT0 

2350 

12487619 

2AD21277 

MV04VJ 

1991 

18302 

8512 

8512 

8521 

41340 

85 

8 

WA4FA0 

2350 

12487619 

2AD21278 

MV04VK 

1991 

18302 

4399 

4489 

4411 

41340 

45 

0 

WA4FA0 

2350 

12487619 

2AD21279 

MV04VL 

1991 

18302 

7940 

7944 

7944 

41340 

8( 

>9 

WAJVEO 

2350 

12487619 

2AD21283 

MV04VQ 

1991 

1  18302 

2575 

4489 

2635 

41340 

2 

37 

WA4FE0 

2350 

12487619 

2AD21284 

MV04VR 

1991 

1  89702 

89702 

89702 

89702 

1  89702 

89 

03 

WAJVBO 

2350 

12487619 

2AD21285 

MV04VS 

1991 

1  18302 

8969 

8969 

9018 

\  41340 

#183 

WAJVAO 

2350 

12487619 

2AD21286 

MV04VT 

1991 

\  18302 

1022 

4489 

1140 

\  41340 

1184 

WAH8AA 

2350 

12487619 

2AD21287 

MV04VU 

1991 

\  18302 

4489 

4489 

772 

\  41340 

/  774 

WAJVAO 

2350 

12487619 

2AD21290 

MV04VX 

1991 

\  18302 

/  6893 

6934 

6957 

W1340 

/  6974 

WA4FE0 

2350 

12487619 

2AD21293 

MV04W0 

1991 

V8302 

f  1104 

4489 

1327 

X340 

/  1327 
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In  Serial  Number  Usage  reports  through  2007,  one  finds  intermittent 
columns  of  identical  odometer  readings — i.e.,  cases  of  multiple  vehicles  in 
different  units  having  the  same  odometer  reading.  This  pattern  could  reflect  an 
error  in  the  LIW  programs  that  process  the  odometer  data.  The  research  team 
did  not  find  this  pattern  in  the  post-2007  data.  However,  analysts  using 
historical  odometer  readings  are  still  likely  to  encounter  data  with  this  quality 
problem. 

Information  about  another  type  of  usage,  fuel  consumption,  can  be  found 
in  OSMIS,  but  this  is  reportedly  a  rough  estimate.  A  TACOM  Cost  and 
Systems  Analysis  interviewee  stated  that  OSMIS  fuel  consumption  figures  are 
“not  truly  representative  of  actual  usage.”  He  remarked  that  “This  is  something 
people  here  [at  TACOM]  have  struggled  with  over  the  years:  We  don’t  know 
how  much  we  are  using  [or]  where.” 
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When  our  research  team  sent  OSMIS  help  staff  (at  Calibre)  questions 
about  the  fuel  consumption  calculation,  the  staff  responded  that  fuel 
consumption  per  vehicle  type  is  “based  on  fuel  consumption  rate  [multiplied 
by]  miles  driven”  by  such  vehicles.  They  also  stated  that  the  fuel  consumption 
rate  for  a  given  system  is  based  on  “information  from  program  managers, 
manufacturers,  actual  data  collection  at  the  unit  level,  Army  publications,  etc.” 
The  fuel  consumption  rates  (gallons  per  mile)  are  not  easy  to  find  (found  in 
OSMIS  Excel  downloads);  are  not  updated  regularly  (April  2009  report  was 
available  as  of  March  2010);  and  are  by  system  (LIN  or  MDS),  rather  than  by 
serial  number.  There  is  no  information  about  the  extent  to  which  the  rates  are 
based  on  actual  consumption  versus  manufacturer  estimates  or  how 
consumption  varies  by  terrain  or  location.13 


13  There  is  a  separate  fuel  report  that  can  be  accessed  in  the  “Weapon  System  Data” 
module  of  OSMIS.  That  report  provides  fuel  cost  per  mile  for  a  given  type  of  vehicle,  and 
OSMIS  support  staff  indicated  that  it  is  derived  by  dividing  vehicle  activity  (miles  driven)  by 
fuel  consumption  (gallons  per  mile),  multiplying  the  result  by  fuel  cost,  and  adding  a  0.02 
percent  surcharge  to  account  for  other  fluids  used  by  vehicles.  This  additional  fuel  report 
does  not  include  fuel  consumption  (gallons  per  mile),  although  the  figure  can  be  derived 
from  the  information  present.  Like  the  fuel  consumption  spreadsheet,  the  fuel  report  data  are 
by  vehicle  type  (LIN  or  MDS),  rather  than  by  serial  number,  and  there  is  no  information 
about  the  extent  to  which  the  cost  per  mile  figures  are  derived  from  actual  consumption 
versus  manufacturer  estimates.  As  of  March  2010,  the  fuel  report  provided  figures  from 
FY95  through  FY08. 
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STAMIS  Lack  Key  Renewal  and  Disposal  Data 
by  Serial  Number 


Data  Element  (by  serial  no.) 

Data  Source 

Ease  of 
Use/ 
Access 

Quality 

History 

»  _  «-» 
5  c)  c 

Repair  time  &  dates 

ILAP 

M 

L 

M 

®  >  re 
u.  _i  S 

Cost:  parts  and  labor 

ILAP 

M 

L 

M 

0> 

o 

GS  repair  time  &  dates 

ILAP 

M 

M 

M 

c  2 

0)  c 

GS  cost:  parts  &  labor 

ILAP 

M 

M 

M 

E  £ 

.5  .£ 
're  're 

Renewal  time,  dates, 
cost  (parts  &  labor) 

AWPS  & 

SDS 

M 

L 

M 

</>  ^ 

Configuration  Mgmt. 

MMIS 

H 

M 

H 

Costs 

None 

— 

— 

— 

<0 

o 

Est.  Cost  to  Replace 

FEDLOG 

H 

M 

— 

Q. 

<2 

Salvage  Value 

None 

— 

— 

— 

o 

Validated  Losses 

AEL  Tracker 

H 

L 

L 
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Assessment  of  Data  on  Maintenance  and  Disposal  of  Army 
Equipment 

Our  review  of  equipment  maintenance  and  disposal  data  in  Army 
STAMIS  suggests  that  maintenance  data  are  moderately  accessible;  are 
low-to-medium  quality;  and  have  moderate  historical  span.  There  is  little 
financial  information  about  disposal  of  Army  equipment.  Disposal  data  that  are 
available  tend  to  be  easy  to  access  but  have  low  quality  and  historical  span. 

Appendix  D  lists  the  basis  for  each  M  and  L  rating,  and  subsequent  slides 
in  this  section  illustrate  key  maintenance  and  disposal  data  quality  issues. 
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Most  Field  Maintenance  Data  Not  Yet  Captured  by  STAMIS 

•  Interviews  with  maintenance  personnel  at  Fort  Campbell 
indicate  work  orders  are  seldom  opened  for  scheduled  field- 
level  maintenance 

•  Interview  with  ILAP  subject  matter  expert  at  Calibre  indicates 

-  SAMS-E  use  has  not  yet  switched  to  two-level  maintenance 

-  Organizational  maintenance  data  capture  only  includes 
deadlining  repairs 

*  EDA  contains  deadlining  organizational  repairs 

•  Some  deadlining  organizational  repairs  (those 
evacuated  to  a  shop)  appear  in  DS  and  GS-DOL 
work  orders 

-  Key  obstacle  to  capturing  all  scheduled  and  unscheduled 
maintenance:  SAMS-E  box  needs  to  allow  multiple  work 
orders  to  be  open  for  same  piece  of  equipment  at  the  same 
time 
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Transitioning  from  Four-Level  to  Two-Level  Maintenance 

As  mentioned  earlier,  the  Army  maintenance  system  has  been  changing 
from  a  four-level  to  a  two-level  structure.  As  described  by  General  Stevenson 
(2002:6),  the  four-level  structure  consisted  of 

•  unit/organizational  level  maintenance  (in  which  equipment  was 
repaired  and  returned  to  the  user) ; 

•  direct  support  (DS)  maintenance  (mostly  repair  and  return  to  the  user, 
but  some  repair  and  return  to  supply); 

•  general  support  (GS)  maintenance  (mostly  repair  and  return  to  supply; 
some  repair  and  return  to  user);  and 

•  depot  maintenance  (repair  and  return  to  supply). 

The  two-level  structure,  which  was  initiated  to  reduce  logistics  footprint  and 
increase  efficiency,  consists  of  field  maintenance  (“on-system  maintenance”  that 
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combines  organizational  and  DS  repairs)  and  sustainment  maintenance  (“off- 
system  maintenance”  that  combines  GS  and  depot  maintenance)  (Stevenson, 
2002:6).  Field  maintenance  includes  organizational  and  support-level  work 
orders. 

As  noted  earlier,  an  ILAP  subject  matter  expert  indicated  that  SAMS-E 
use  has  not  yet  transitioned  from  four-level  to  two-level  maintenance. 
Additional  information  about  the  transition  status  appears  in  a  PM  LIS  briefing 
(2009:9): 

Today  SAMS-E  provides  limited  support  to  a  full  integration  to 
Two-Level  Maintenance  because  of  the  way  it  is  programmed; 
however,  these  limits  are  set  and  perpetuated  by  a  current  mix  of 
legacy  and  SAMS-E  systems  in  the  field,  thus  generating  a 
requirement  to  continue  to  report  the  equipment  readiness  to  LIW 
in  the  traditional  way  and  format.  Units  are  still  expected  to  report 
Direct  Support  and  Organizational  data  separately  while  the  Army  is 
directed  to  operate  and  practice  only  Field  and  Sustainment 
Maintenance. 

Thus,  given  that  there  is  currently  a  two-level  maintenance  system  but  four- 
level  maintenance  reporting  and  data  capture,  we  refer  to  field/sustainment  as 
well  as  organizational,  DS,  and  GS-DOL  maintenance  in  the  paragraphs  that 
follow. 

Field-Level  Maintenance  Data 

A  key  gap  in  maintenance  data  is  that  there  are  few  records  of  scheduled 
field  maintenance.  A  RAND  researcher  who  interviewed  maintenance 
personnel  at  Fort  Campbell  stated,  “Maintainers  at  Campbell  say  it  would  be 
very  unusual  for  any  shop  to  open  a  work  order  for  scheduled  [field-level] 
maintenance”  (Lackey,  2010). 

Another  field-level  maintenance  data  gap  in  STAMIS  is  a  lack  of  records 
of  nondeadlining  organizational  maintenance.  The  Equipment  Downtime 
Analyzer  (EDA)  captures  deadlining  organizational  repairs  (Peltz,  Robbins, 
Boren,  and  Wolff,  2002;  Girardini,  Lackey,  and  Peltz,  2007). 
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GS-DOL  Data  Quality  (Archived  in  ILAP)  Has  Been  Improving 


Missing  data  percentages  have  decreased  from  2000  to  2009 

GS-DOL  completed  work  order  data  for  the  five  ground  system 
FSCs  with  the  most  work  orders:  2320,  2330,  2350,  2310,  2420 


Year 

#  WOs 

Percent 

Serial  Numbers 
Missing 

Percent 
Fault  Data 
Missing 

Percent 
Cost  Data 
Missing 

2000 

7,698 

63.03 

6.95 

37.11 

2001 

45,421 

65.68 

9.2 

64.04 

2002 

57,396 

40.73 

19.51 

37.35 

2003 

65,305 

36.64 

4.26 

22.59 

2004 

73,840 

26.04 

0.02 

8.75 

2005 

102,893 

36.67 

0.06 

5.35 

2006 

65,038 

18.03 

0.17 

4.14 

2007 

168,317 

4.14 

1.46 

5.58 

2008 

119,305 

1.13 

1.15 

4.98 

2009 

159,121 

0.05 

0.91 

7.07 
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Sustainment-Level  Maintenance  Data 

The  quality  of  sustainment-level  maintenance  data  has  improved 
markedly.  Almost  all  GS-DOL  work  orders  currently  include  serial  numbers 
and  descriptions  about  the  fault  that  led  to  the  repair,  and  only  7  percent  are 
missing  data  on  labor  and  parts  costs.  Work  orders  from  earlier  years  are 
missing  considerably  greater  percentages  of  serial  numbers,  fault  information, 
and  costs. 
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Renewal  Dates  and  Costs  by  Serial  Number  Are  Not 

Available  in  STAMIS 

•  No  central  repository  of  renewal  actions 

•  Equipment  Control  Records  in  TEDB  do  not  archive  renewal  dates 

•  Gathering  renewal  data  from  non-STAMIS  sources  is  “hit  or  miss” 

Equipment  Control  Record 


Control  Number: 

Organization: 

Location: 

0142 

ROGERS  G1 

1ST  ON  1420  MU  D  ARTILLERY  BTRY  C 

ARKANSAS 

Unit  Ident  Code: 

Utilization  Code: 

Vehicle  Use  Code: 

m  i  •  a 

|2 

.HO* 

Nomenclature: 

Model: 

NSN: 

TRUCK  CARGO  10  TON 

M983 

237001 100/623 

Serial  Number: 

Registration  Number: 

Year  of  Mfg: 

10T7KM7XD1021037 

NP06D4 

1903 

Manufacturer: 

Contract  Number: 

PO  Number: 

blfi 

DAAE0/  SIC -5260 

Warranty  Period: 

Report  Type: 

Report  Code: 

TRANSFER 

2 

Shipped  To  UIC: 

Received  From  UIC: 

W0L6AA 

Remark# : 

Authorizing  Official: 

Updated  By:  Trans  Date: 

Recon  By: 

Recon  Date 

Ne  Autheil/lng  Official 

Updated  by  the  Legacy  Syetem  31  Jul  93 

CWJ  Al  AN  L  CAMERON 
COM:  87024106 36 

24  Jul  08 

AlaM.I.caineioiftVr'u*  auny.mil 

1ST  ON.  142ND  J  A 
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Additionally,  Army  STAMIS  generally  do  not  contain  data  on  equipment 
renewal  by  serial  number.  AWPS  and  SDS  contain  data  on  aggregate  data  on 
depot  maintenance.  These  include  the  quantity  of  items  inducted  with  a  given 
Procurement  Request  Order  Number  (PRON)  and  NSN;  the  quantity 
completed;  their  average  repair  cycle  time;  and  their  total  labor,  material,  and 
other  costs.  Such  data  are  used  for  workload  planning,  determining  manpower 
requirements,  and  justifying  budget  requests  (Warren,  2002b).  However,  they 
are  not  sufficient  for  linking  the  renewal  history  of  vehicles  to  the  performance 
and  subsequent  sustainment  costs  of  those  vehicles  or  for  determining  the 
optimal  timing  of  renewal  in  the  life  of  a  system. 

Although  depots  are  expected  to  send  renewal  dates  by  serial  number  to 
LOGSA  via  DA  Form  2408-9,  a  LOGSA  interviewee  indicated  that  depots 
seldom  send  this  information.  In  addition,  when  such  data  do  reach  LOGSA, 
they  do  not  appear  in  the  Equipment  Control  Record  (ECR).  As  shown  in  the 
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above  slide,  the  ECR  has  no  field  showing  the  renewal  date.  LOGSA  stores  any 
renewal  data  it  receives  (via  Form  2408-9)  in  a  Recap-Rebuild-Overhaul  table 
that  is  accessible  to  LOGSA  but  not  others.  LOGSA  sent  the  table  to  our 
research  team,  and  most  entries  were  overhauls  that  occurred  prior  to  1998. 
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So  Far,  the  New  Army  Equipment  Loss  (AEL)  Tracker 
Database  Has  Quality  Limitations  but  Good  Historical  Span 


Data  Element 

%  Missing  (quality) 

Time  Period  (breadth) 

Loss  date 

0%  missing  (good  quality) 

09MAY85  -  16DEC09 

Validation  date 

99.99%  (poor  quality) 

— 

Incident  type 

84% 

— 

Incident  date 

99% 

14JUN99  -  29SEP09 

Incident  location 

99% 

— 

Disposition  date 

98% 

01OCT01  -01AUG09 

Disposition  category 

96% 

— 

Attrition  date 

83% 

26DEC01  -  060CT09 
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Disposal  Data 

According  to  interviewees  from  the  TACOM  Cost  and  Systems  Analysis 
department,  there  is  a  lack  of  data  related  to  the  demilitarization  of  equipment 
in  the  disposal  process.  As  one  such  analyst  noted, 

A  missed  data  collection  effort  is  demilitarization.  When  you 
dispose  of  a  vehicle,  there  are  certain  parts  that  you  have  to  break 
down  in  certain  ways.  [There  are]  environmental  costs, 
transportation  costs — a  whole  process  that  occurs  when  a  vehicle  is 
demilitarized.  To  my  knowledge,  there  isn’t  a  data  source  for  actual 
demilitarization  costs  .  .  .  and  those  can  be  significant  costs.  There 
are  agencies  (like  the  Army  Environmental  Agency)  that  will  develop 
environmental  estimates  for  programs,  but  the  estimates  represent 
mostly  rough-order-of-magnitude  estimates  based  on  expert 
opinion. 
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In  general,  Army  STAMIS  contain  little  information  about  equipment  disposal. 
Salvage  values  and  disposal  cost  data  are  not  available.  However,  LOGSA  did 
recently  add  the  Army  Equipment  Loss  (AEL)  Tracker  to  LIW. 

The  AEL  Tracker  contained  27,043  records  as  of  January  4,  2010.  As 
shown  in  the  above  slide,  each  record  had  the  date  of  the  loss  but  typically  did 
not  have  additional  information  about  the  loss.  For  instance,  26,038  records 
(96  percent  of  the  total)  did  not  have  the  disposition  category  (“DRMO,” 
“Dispose  in  Place,”  “Pending  Legal  Determination,”  “Center  for  Military 
History”)  listed,  and  99  percent  did  not  state  the  location  of  the  incident  that 
led  to  the  loss.  Although  22,748  records  (84  percent  of  the  total)  listed  the 
incident  type,  for  21,361  of  those  records  the  incident  type  was  “21,”  which 
translates  to  “Other-Tell  LOGSA  to  add  new  type  incident.”  The  quality  of 
AEL  Tracker  data  may  soon  improve,  however.  Materiel  managers  are  in  the 
process  of  gaining  special  System  Access  Request  permissions  to  update  AEL 
Tracker  information.  Once  they  are  able  to  update  the  database,  more  detailed 
equipment  loss  data  may  be  available. 
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Key  LCM  Decisions  Are  Hindered  by  Data-Quality  Issues 


RAND 
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Impact  of  Data  Limitations 

After  evaluating  LCS  data  elements  available  in  Army  STAMIS,  we 
returned  to  the  tables  (found  earlier  in  this  document)  that  identify  (with  an  X) 
each  data  element  needed  for  analyses  supporting  key  LCM  decisions.  We  then 
shaded  cells  of  the  needed  data  elements  based  on  their  quality  ratings:  green 
for  high,  yellow  for  medium,  or  red  for  low.  Most  cells  in  the  above  table  were 
shaded  red  or  yellow,  an  indication  that  existing  data  are  not  sufficient  for 
effective  analyses,  and  empirically  well-supported  decisions,  in  the  areas  of 
acquisition  strategy,  upgrade  planning,  renewal  planning,  and  scheduled  service 
updates. 
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Key  LCM  Decisions  Are  Hindered  by  Data  Quality  Issues 


RAND 
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Similarly,  in  the  above  table,  which  identified  decision  elements  needed 
for  four  resource/budget  planning  and  materiel  development  decisions,  most 
cells  were  either  yellow  (medium  quality)  or  red  (low  quality).  That  is,  existing 
data  also  do  not  facilitate  decisions  and  analyses  related  to  maintenance 
workforce  planning,  budgeting,  new  system  design,  and  system  performance. 
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Interviews  Provided  Examples  of  Insufficient  LCS  Data 
Impeding  LCM  Analyses 

•  Decision:  Determining  new  system  design  and  acquisition 
requirements  to  ensure  supportability 

-  Analyses:  Drivers  of  scheduled  and  unscheduled  maintenance  time 

-  LCS  data  problem:  Lack  of  work  orders  with  labor  hours  for 
field-level  scheduled  maintenance 

•  Decision:  Whether  to  upgrade  a  component  (e.g.,  wiring  harness) 

-  Analyses:  Assess  part  reliability,  costs,  primary  reasons  for  failure 

-  LCS  data  problem:  Work  order  data  gaps  -  i.e.,  part  repair/ 
replacement  by  vehicle  serial  number  and  reason  for  demand 

•  Decision:  Resource  needs  for  fleet  replacement  &  recapitalization 

-  Analyses:  Analyses  of  vehicle  aging  effects,  recapitalization 
cost-per-vehicle  trends 

-  LCS  data  problem:  Missing  historical  repair/replacement  of  vehicle 
components,  renewal  dates,  and  types;  missing  vehicle-level 
recapitalization  costs 
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Interviewees  described  the  effects  of  LCS  data  gaps  on  critical  analyses  and 
decisions.  A  LOGSA  manager  (Table  2.2(a))  discussed  how  maintenance  data 
shortcomings,  including  insufficient  scheduled  maintenance  data,  impact 
decisions  and  plans  related  to  the  design  and  acquisition  of  new  systems. 

Another  interviewee  (from  TACOM  Cost  and  Systems  Analysis 
department)  noted  that  a  lack  of  information  about  the  reasons  for  part 
demands — whether  a  part  failed  because  of  an  accident  for  instance — makes  it 
difficult  to  support  decisions  about  whether  or  not  to  upgrade  parts.  The 
LOGSA  manager  made  a  similar  observation  (Table  2.2(b)). 

Additionally,  a  TACOM  Cost  and  Systems  manager  described  several 
impediments  to  determining  resource  needs  for  replacement  and 
recapitalization  of  vehicles  in  Army  fleets.  He  noted  that  an  obstacle  to  assessing 
aging  effects  (when  replacement  and/or  recapitalization  is  needed)  is  the  lack  of 
historical  data  on  the  replacement  of  vehicle  components  (Table  2.2(c)).  He 
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also  noted  that  an  obstacle  to  tracking  recapitalization  costs  per  vehicle  is  the 
lack  of  vehicle-level  renewal  data  (Table  2.2(d)). 

A  manager  at  HQDA  G-4  pointed  out  that,  in  general,  frequent  questions 
about  the  integrity  of  the  data  lead  analysts  in  his  organization  to  spend  extra 
time  “revisiting  grounds  to  validate  the  data.”  There  is  also  “a  constant  fear” 
that  erroneous  data  will  be  presented  to  leadership  (Table  2.2(e)). 
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Table  2.2 

Interviewee  Statements:  Effects  of  LCS  Data  Gaps 


Interviewee 

Statement 

(a) 

LOGS  A 
manager 

"Legacy  data  from  a  previous  system  is  the  best  thing  to  have.  You  want 
to  know  what  were  the  maintenance  drivers.  For  example,  how  many 
maintainers  did  we  have  for  a  fleet  of  50  vehicles?  What  did  they  spend 
time  on — oil  changes?  Why  were  oil  changes  taking  all  their  time? 
Whatever  these  maintenance  folks  are  doing,  we  need  to  look  at  how 
to  reduce  their  time  on  the  next  vehicle.  Having  data  on  a  predecessor 
system  during  acquisition  is  very  important." 

(b) 

LOGS  A 
manager 

"There  was  a  wiring  harness  on  one  ground  vehicle  that  was  very 
expensive  and  very  time-consuming  to  replace.  When  the  turret  opening 
came  down  a  certain  way,  it  would  pinch  the  wiring  harness.  Wiring 
harnesses  were  failing  because  a  clip  was  broken.  The  vehicle  was 
[ostensibly]  down  because  of  this  wiring  harness,  but  it  really  did  not 
have  to  do  with  the  wiring  harness;  it  was  because  the  clip  was  loose.  So 
how  do  you  get  accurate  information  about  the  cause  of  the  failure?" 

(c) 

TACOM  Cost 
and  Systems 
Analysis 
manager 

"One  of  the  things  we've  wanted  to  look  at  but  haven't  been  able  to 
track  is  costs  with  age.  You  may  have  a  vehicle  that  is  a  composite  of  an 
engine  5  years  old,  a  suspension  15  years  old.  ...  So  when  people  want 
to  look  at,  over  time,  what  age  does  to  sustainment  costs  of  the  Army — 
well,  what  age  are  you  talking  about?" 

(d) 

TACOM  Cost 
and  Systems 
Analysis 
manager 

"Leadership  came  to  us  and  asked  us  to  track  the  improvement  in  unit 
[recap]  cost  over  time,  since  the  initial  unit  cost  exceeded  the  [target]. 
There  wasn't  a  reporting  system  we  could  readily  go  to  within  the 
depots  to  be  able  to  identify  the  labor  and  material  required  to  do  the 
work  on  one  vehicle.  They  knew  how  much  money  they  spent  per 
month  and  how  many  vehicles  they  had  recapitalized  in  the  month — but 
did  not  have  the  data  at  the  vehicle  level,  which  made  it  challenging  to 
see  the  unit  cost  improvement  over  time.  The  information  systems  that 
were  at  the  depot  .  .  .  we  couldn't  get  access,  and  once  we  got  access, 
we  found  significant  [data]  shortfalls.  [To  get  access]  it  took  the  CG 
[Commanding  General]  going  to  the  depots  to  say,  'Let  my  cost  guys  in, 
and  give  them  access  to  your  cost  and  labor  data.'" 

(e) 

HQDA  G-4 
manager 

"If  we  rely  on  data  that  later  proves  to  be  inaccurate  and  provide  it  to  a 
3-star  or  CSA  [Chief  of  Staff  of  the  Army]  level  or  Congress,  then  .  .  . 
when  the  data  go  out  it's  hard  to  reel  back  in.  Hard  to  go  back  and 
reverse  something  if  the  data  were  in  error." 
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Lack  of  STAMIS  Reset,  Recap,  and  Overhaul  Data  Impedes 
Assessment  of  Army  Equipment  Renewal  Program 

RAND  team  faced  significant  obstacles  in  recent  effort  to  gather  renewal  data 

•  Needed  renewal  (reset/recap/overhaul/refurb)  dates  by  serial  number 

•  Data-gathering  process  required  multiple  phone  calls  and  emails  to 
scattered  non-STAMIS  sources 

-  Some  PMs  keep  their  own  databases  with  renewal  data  by  serial 
number  and  provided  the  data  they  had  available 

-  Other  PMs  needed  to  send  special  requests  for  data  to  contractors 
and  depots 

-  Reset,  recapitalization,  and  theater  provided  equipment 
refurbishment  (TPER)  data  each  came  from  different  POCs 

-  Data  from  different  sources  had  different  fields  (content  and  formats) 

•  Renewal  cost  by  serial  number  available  for  CE/MHE  but  not 
tactical  vehicles 

•  Different  dates  encountered  in  different  datasets:  “acceptance 
date”;  “start  date”  and  “end  date”;  “date  returned”;  “DD250/ship 
date” 

•  Renewal  data  for  some  systems  were  not  provided  (or  do  not  exist) 
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RAND  researchers  conducting  a  study  to  assess  equipment  renewal  effects 
also  found  that  insufficient  STAMIS  data  on  reset,  recapitalization,  overhaul, 
and  refurbishment  was  a  significant  impediment,  greatly  increasing  the  time 
needed  to  conduct  analysis,  the  resources  needed,  and  sometimes  limiting  the 
analytic  techniques  that  could  be  employed.  As  outlined  in  the  above  slide,  the 
study  called  for  renewal  history  (renewal  dates)  by  serial  number  for  multiple 
types  of  vehicles,  including  tactical  vehicles,  force  projection  vehicles,  and 
combat  vehicles.  Because  there  was  not  a  STAMIS  source  of  such  serial-level 
data,  the  research  team  attempted  to  gather  the  data  from  non-STAMIS 
sources. 

Gathering  these  non-STAMIS  data  entailed  emailing  and  calling  multiple 
program  managers  (PMs)  and  other  contacts  at  TACOM  to  request  data  for 
different  types  of  vehicles.  The  team  contacted  managers  at  PM  Bradley 
regarding  Bradley  Fighting  Vehicle  data;  PM  Heavy  Tactical  Vehicles  (HTV) 
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regarding  HTV  data;  and  PM  Force  Projection  regarding  combat 
engineer/material  handling  equipment  (CE/MFIE)  data.  In  some  cases, 
different  managers  in  the  same  PM  office  (PM  FiTV)  were  responsible  for 
overseeing  reset,  recapitalization,  and  theater-provided  equipment 
refurbishment  of  a  given  type  of  vehicle.  The  team  needed  to  send  emails  and 
make  phone  calls  to  many  of  these  managers  to  describe  the  study,  explain  the 
need  for  the  data,  and  follow  up  on  data  requests. 

PMs  are  frequently  approached  with  requests  for  renewal  data.  Each 
request  requires  the  PM  to  verify  the  identity  and  authorized  need  of  each 
requestor,  determine  the  time  span  and  data  fields  needed,  and  then  prepare 
and  send  files  to  meet  those  needs.  Given  the  many  responsibilities  of  PMs  and 
their  varying  approaches  to  tracking  renewals,  it  can  be  difficult  to  respond  to 
such  requests.  Some  managers  were  able  to  provide  the  data  promptly  (within  a 
week),  and  others  sent  the  data  after  a  few  months.  The  data  received  varied  in 
content  (different  types  of  dates  listed)  and  format  (PDF  files,  single  Excel 
spreadsheet,  multiple  Excel  spreadsheets).  For  some  systems,  renewal  data  were 
not  provided.  These  problems  limited  the  number  of  systems  for  which  renewal 
effects  could  be  assessed.  Ensuring  that  STAMIS  archive  such  data  will  reduce 
the  burden  on  PMs  and  facilitate  assessment  of  renewal  program  value. 
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Army  Policy  Is  Not  Well-Aligned  with  Data  Needs 
for  LCS  Analyses  and  Decisions 

•  Policy  of  depending  on  units  for  manual  input  and  data  quality 

-  “Units  perform  quarterly  reconciliations  between  equipment  data  shown  in 
units’  ULLS-G  equipment  data  file  and  equipment  data  shown  on  unit  property 
books  to  ensure  data  synchronization  and  accuracy.” 

(DA  PAM  750-8:133) 

•  Policy  only  requires  organizational  work  orders  for  NMC  repairs 
(DA  PAM  750-8:35) 

•  No  policy  for  STAMIS  to  get  and  archive  renewal  data  (dates,  types,  costs) 
by  serial  number 

-  Policy  refers  to  AMC  archive:  “Historical  information  on  depot  maintenance 
operations  will  be  archived  by  AMC  and  provided  on  demand  to  appropriate 
users.”  (AR  750-1:  16) 

•  Dependence  on  PMs  to  ensure  renewal  data  are  gathered  and  analyzed 

-  Materiel  Developers  (MATDEVs)  will  “Ensure  that  data  collected  from  all  levels 
of  maintenance  are  analyzed  and  used  for  reliability  improvement  and 
updating  logistical  and  manpower  data  bases  usecf  in  determining  Army  force 
structure  maintenance  requirements....”  (AR  750-1:7) 

-  “MATDEVs  will  track  efficiency  programs  (for  example,  Recapitalization)  to 
validate  cost  avoidance.”  (AR 750-1:77) 

-  “the  MATDEV  will . . .  acquire  all  logistics  data  relevant  to  the  materiel  system” 
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Factors  Contributing  to  Data  Issues 

A  review  of  relevant  literature  and  interview  statements  suggests  that  three 
sets  of  factors  contribute  to  the  aforementioned  data  issues  and  their  effects:  (1) 
policy  factors  (limitations  of  existing  Army  maintenance  policies);  (2)  design 
factors  (information  system  design  features);  and  (3)  execution  factors  (how 
Army  doctrine  and  policy  are  carried  out  by  personnel). 


Policy  Factors 

The  primary  Army  policy  documents  related  to  maintenance  and  usage 
data  are  DA  PAM  750-8,  The  Army  Maintenance  Management  System 
(TAMMS)  Users  Manual  (HQDA,  2005b),  and  AR  750-1 ,  Army  Materiel 
Maintenance  Policy  (HQDA,  2007a).  Portions  of  both  are  not  conducive  to 
driving  the  capture  and  archiving  of  accessible,  high-quality,  longitudinal  LCS 
data  in  STAMIS.  DA  PAM  750-8  places  the  burden  on  units  to  keep  track  of 
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instructions  for  manually  entering  equipment  maintenance  and  usage 
information  either  via  the  Army  Materiel  Status  System  (AMSS)  end-of-period 
report  process,  electronic  forms,  or  paper  forms;  it  also  relies  heavily  on  units  to 
review  reports  and  identify/ correct  inaccuracies. 

Additionally,  DA  PAM  750-8  specifies  that  an  organizational  work  order 
number  (ORG  WON)  is  assigned  to  track  (a)  inoperative,  not  mission  capable 
(NMC)  equipment  that  is  repaired  by  the  unit;  and  (b)  any  equipment  sent  to  a 
support  maintenance  activity  for  repair.  Consequently,  when  units  repair 
nondeadlined  (still  operating)  items,  records  of  such  repairs  do  not  reach  ILAP. 

Gaps  in  AR  750-1  may  be  contributing  to  the  lack  of  a  STAMIS  source  of 
renewal  data  by  serial  number.  Although  the  policy  refers  to  gathering  and 
archiving  depot  maintenance  data,  it  does  not  specify  that  the  data  should  be  at 
the  serial  level.  Moreover,  by  stating  that  AMC  should  provide  depot 
maintenance  information  “on  demand  to  appropriate  users,”  the  policy  does 
not  ensure  accessibility  of  the  data  via  STAMIS. 

The  role  of  the  Materiel  Developers  (MATDEVs) — principally,  the  Army 
Program  Executive  Offices  (PEOs)  and  Program  Managers  (PMs) — figures 
prominently  in  AR  750-1.  Among  the  designated  MATDEV  responsibilities  is 
ensuring  that  renewal  data  are  gathered  and  analyzed  so  that  total  ownership 
cost  can  be  reduced  throughout  a  weapon  system’s  life  cycle.  For  programs  such 
as  recapitalization,  the  MATDEV  is  expected  to  have  a  “detailed  evaluation 
plan  covering  data  collection  plan  measures  of  success.”  Interviewees  expressed 
concerns  about  this  reliance  on  MATDEVs,  noting  that  PMs  are  still  more 
focused  on  acquisition  than  on  sustainment  because  they  are  (1)  typically  in 
their  positions  for  only  three  years;  (2)  still  have  more  control  over  acquisition 
funds  than  sustainment  funds;  and  (3)  are  pushed  to  get  new  systems  fielded 
quickly. 

Along  the  same  lines,  a  recent  GAO  report  (Solis,  2009:15)  suggests  that 
PMs  may  not  be  devoting  sufficient  attention  to  equipment  sustainment.  In 
particular,  PMs  have  not  been  involving  depot  officials  in  “the  sustainment 
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portion  of  the  life  cycle  management  planning  process  for  new  and  modified 
systems”  because  there  is  no  formal  requirement  for  PMs  to  do  so. 
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Design  and  Execution  Factors  Contribute  to  Data  Issues 

•  Design  Factors 

-  Too  much  manual  input 

•  Age,  usage,  renewal  data 

-  Limited  error-  and  mistake-proofing  (screening  and  rejecting)  of 
inaccurate/invalid  inputs  to  databases 

-  Some  data  capture  not  properly  designed  into  systems 

•  Each  Equipment  Control  Record  overwrites  the  previous  one  in  TEDB 

•  Serial  number  field  in  maintenance  systems  limited  to  15  digits 

•  Execution  Factors 

-  Perception  that  data  reporting  is  not  needed  while  deployed 

•  Lack  of  enforcement  of  data  reporting  requirements 

-  Insufficient  securing  of  access  to  contractor  data 

-  Serial  numbers  change  with  recapitalization  even  though  DA  PAM  750-8 
states  they  should  not 
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Design  Factors 

In  addition  to  policy  factors,  design  factors  may  be  contributing  to  LCS 
data  problems.  Due  to  system  changes  over  the  past  several  years,  units  can  now 
input  manufacture  dates,  odometer  readings,  and  maintenance  data  (including 
renewal  dates)  via  electronic  forms  rather  than  hard  copies  (DA  Form  2408-9 
on  LIW  and  the  Unit  Level  Logistics  System-Ground  (ULLS-G)/Army 
Materiel  Status  System  (AMSS)  monthly  reports).  (Some  units  do  not  have 
ULLS-G  or  LIW  access,  so  hard  copies  of  forms  such  as  the  “Installation 
Materiel  Status  Report”  are  still  accepted.)  Even  with  electronic  forms, 
however,  data  are  entered  manually  and  are  subject  to  keystroke  errors,  logic 
errors  (misinterpreting  a  question  asked),  or  similar  problems.  An  unofficial 
UIC — one  created  for  local  use  rather  than  one  recognized  by  the  Army  Status 
of  Resource  and  Training  System  (ASORTS)  or  the  wrong  reporting  date  can 
lead  to  a  flawed  report,  which  AMSS  treats  as  a  missing  report  (LOGSA,  2007). 
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Also,  opening  an  AMSS  report  in  Word  or  Excel,  instead  of  Notepad, 
generates  characters  in  the  file  that  skew  the  data  submitted  to  LOGSA 
(Williams,  2009). 

The  research  team  noted  several  opportunities  for  keystroke  and  logic 
errors  in  electronic  DA  Form  2408-9.  Specifically,  the  user  needs  to  remember 
to  select  “overhaul”  from  the  report  code  menu  if  the  form  is  being  used  to 
submit  overhaul  data,  and  the  user  then  needs  to  enter  overhaul  year  in  the 
“Year  of  Manufacture”  field  using  a  special  format  (lH05=first  overhaul 
performed  in  2005).  Users  could  also  potentially  select  incorrect  items  such  as 
the  utilization  code  from  several  other  menus.  Although  ULLS-G/AMSS  is 
normally  used  to  report  both  readiness  and  usage  data,  DA  Form  2408-9  has  a 
field  for  “Usage  Miles”;  the  user  is  supposed  to  enter  the  current  odometer 
reading  in  that  field  but  could  inadvertently  enter  miles  traveled  (usage)  since 
the  last  reading. 

Commercial  information  systems  often  use  two  approaches  for  improving 
the  quality  of  manually  entered  data:  (1)  error-proofing  (preventing  an  error); 
and  (2)  mistake-proofing  (detecting  and  correcting  an  error).  In  electronic  DA 
Form  2408-9,  we  found  some  error  proofing:  warning  messages  that  indicated 
when  an  invalid  UIC,  NSN,  or  serial  number  was  entered.  Invalid  dates  and 
odometer  readings  are  not  rejected,  however.  A  recent  LOGSA  briefing 
(Williams,  2009)  cautioned  that  entering  the  tenths  digit  on  an  odometer 
reading — 3000.5  instead  of  3000  or  3001 — in  ULLS-G  will  skew  the  usage 
report.  Along  the  same  lines,  a  LOGSA  interviewee  noted  that  “there  are  no 
up-front  edits  in  PBUSE,  so  a  PBO  [property  book  officer]  can  put  in  anything 
he  wants  to — duplicate  serial  numbers,  duplicate  registration  numbers.  .  .  ” 

Another  information  system  feature  contributing  to  data  issues  is  that 
some  data  capture  is  not  designed  into  systems.  New  data  submitted  on  DA 
Form  2408-9  can  overwrite  existing  data  on  the  TEDB  Equipment  Control 
Record.  As  a  result,  some  vehicle  ECRs  list  recapitalization  dates  as  the  year  of 
manufacture.  Also,  a  LOGSA  manager  noted  that  maintenance  and  supply 
systems  do  not  interface  and  have  discrepancies  in  their  data  fields — the  serial 
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number  field  is  20  characters  in  PBUSE  and  LIW  but  is  only  15  characters  in 
maintenance  systems.  Some  serial  numbers  exceed  1 5  characters  and  are  cut 
short  in  maintenance  databases  due  to  the  field  length  restriction. 

Execution  Factors 

Poor  execution  of  current  Army  policies  contributes  to  LCS  data 
problems  as  well.  Although  Army  policy — specifically,  DA  PAM  750-1 
(HQDA,  2007b:  19) — states  that  readiness  reports  “will  be  completed  during 
both  peacetime  and  military  operations,”  interviewees  pointed  out  that 
deployed  units  tend  to  perceive  such  reports  as  optional  (Table  2.3(a)). 
Interviewees  also  suggested  that  lack  of  enforcement  of  the  policy  leads  to 
missing  usage  and  readiness  reports  from  deployed  units  (Table  2.3(b)). 

They  also  noted  that  the  readiness  reporting  policy  is  not  well-executed, 
and  interviewees  stated  (Table  2.3(c))  that  policies  related  to  contractor 
maintenance  data  are  not  being  fully  observed.  Such  policies  require  contractors 
performing  Army  equipment  maintenance  to  “report  weekly  open  and  closed 
work  order  production  .  .  .  information  to  LOGSA”  (AR  750-1,  HQDA, 
2007a:43)  and  require  that  contracts  “include  provisions  for  the  collection  of 
work  order  .  .  .  data  from  the  contractor”  (AR  750-1,  HQDA,  2007a: 44). 

When  examining  recapitalization  data  provided  by  Program  Managers,  we 
found  that  another  policy  is  not  being  observed  regularly: 

Serial  numbers  that  have  been  assigned  to  an  item  are  not  changed 
during  its  life  cycle,  regardless  of  changes  in  configuration.  The 
exception  to  this  rule  is  for  the  correction  of  errors  resulting  in 
duplication  of  numbers  (DA  PAM  750-8,  HQDA,  2005b:  134). 

Despite  this  policy,  recapitalized  vehicles  are  often  assigned  new  serial  numbers. 
Because  STAMIS  do  not  contain  such  old-to-new  serial  number  translations,  it 
is  difficult  to  assess  the  effect  of  recapitalization  on  vehicles — in  order  to  track 
their  failure  rates  before  and  after  recapitalization — without  having  PM 
spreadsheets  that  show  the  correspondence  between  old  and  new  serial 
numbers. 
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Table  2.3 

Interviewee  Statements:  Reasons  for  LCS  Data  Problems 


Interviewee 

Statement 

(a) 

HQDA  G-4 
manager 

"When  you  have  units  stateside,  they  provide  data;  when  you  have 
deployed  units,  it's  harder  to  get  the  reports  .  .  .  You  have  a  lot  of 
commanders  in  the  Army  who  were  in  Desert  Shield.  We  waived 
reporting  requirements  during  that  time.  Now  it's  a  protracted  conflict, 
and  Congress  is  asking  questions  about  the  impact  on  equipment  of 
prolonged  conflict.  But  [units]  don't  understand  downstream  that 
there  is  a  real-time  requirement  for  information." 

(b) 

TACOM  ILSC 
manager 

"In  peacetime,  reporting  is  more  accurate;  in  a  deployed  environment, 
it's  less  accurate.  What  are  you  going  to  do — call  [a  general]  and  chew 
him  out  because  82nd  Airborne  didn't  report?  How  do  you  enforce  it?" 

(c) 

ASA[ALT] 

manager 

"CLS  [Contractor  Logistics  Support]  creates  burdens  on  the  process 
because  the  visibility — to  the  same  level  that  [HQDA  G-4]  may  desire  to 
have  (and  that  they  do  have  on  other  programs)  is  not  resident  on  the 
Stryker  program.  It's  data  that  the  G-4  is  getting  more  visibility  on,  but 
since  they  don't  control  the  databases,  they  don't  necessarily  have  the 
same  flexibility  to  manipulate  the  data  as  they  need  to  answer 
particular  questions." 
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Impact  of  GCSS-A  and  LMP  on  Data  Access 
and  Quality  Not  Yet  Clear 

•  GCSS-A  -  Tactical-level  integrated  logistics  system  architecture 

-  Purpose:  To  integrate  supply,  maintenance,  property,  ammunition,  and 
finance  data  in  a  single  database  accessible  to  all  authorized  users; 
will  replace  legacy  STAMIS 

-  Concerns  raised: 

•  Will  business  rules  and  processes  support  enterprise 
requirements? 

•  Will  data  quality  issues  in  STAMIS  carry  over? 

-  Disposal  cost  capture  unclear 

-  Initial  data  cleansing  may  not  be  sufficient 

•  LMP  -  National-level  integrated  logistics  system  architecture 

-  Purpose:  To  modernize  wholesale  materiel  and  maintenance  systems, 
(1)  Commodity  Command  Standard  System  (CCSS)  and  (2)  Standard 
Depot  System  (SDS) — the  data  source  for  AWPS 

-  Concerns  raised: 

•  Depot  maintenance  data  not  at  serial  number  level 

-  does  not  address  need  for  renewal  data  archiving 
'  Disruptive  transition  so  far 
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Potential  Impact  of  STAMIS  Changes  in  Progress 

Over  the  past  decade,  several  programs  were  initiated  to  change  STAMIS 
structure  and  content.  The  Global  Combat  Support  System-Army  (GCSS-A)  is 
an  automated  logistics  system  that  uses  the  SAP  Defense  Forces  and  Public 
Security  (DFPS)  software  platform  to  provide  Enterprise  Resource  Planning 
(ERP).  It  is  designed  to  integrate  tactical  logistics  data  and  functions  as  related 
to  supply,  maintenance,  property,  ammunition,  and  finance  that  presently 
reside  in  multiple  STAMIS  (PBUSE  and  SAMS-E).  GCSS-A  is  still  undergoing 
operational  assessment  and  has  not  yet  been  fielded. 

The  Logistics  Modernization  Program  (LMP)  focuses  on  national-level 
(wholesale)  logistics,  integrating  inventory  management  and  depot  operations 
data  that  currently  reside  in  several  legacy  STAMIS:  Commodity  Command 
Standard  System  (CCSS)  and  Standard  Depot  System  (SDS).  Like  GCSS-A, 
LMP  leverages  SAP  software.  LMP  and  GCSS-A  will  be  key  components  of  the 
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Single  Army  Logistics  Enterprise  (SALE),  a  network  of  automated  logistics 
information  systems  linked  to  provide  end-to-end  visibility  of  national  and 
tactical  Army  logistics  (Anderson,  2009). 

Subject  matter  experts  (Egan,  2009)  indicate  that  GCSS-A  and  LMP  have 
the  potential  to  address  some  of  the  data  issues  discussed  in  this  document.  First, 
as  part  of  SALE,  they  may  contribute  to  better  asset  visibility,  including  more 
accurate  information  about  fleet  sizes  and  equipment  on-hand  quantities  in  units 
or  regions.  Second,  with  newer  computers  and  software  replacing  outdated  ones, 
run  times  for  operations  and  maintenance  reports  may  be  faster.  Third,  with 
more  centralized  databases,  LCS  data  accessibility  is  likely  to  improve. 

However,  interviewees  raised  several  concerns  that  suggest  SALE  should  not 
be  seen  as  a  cure-all  for  LCS  data  issues.  The  primary  caveats  are  outlined  below. 

GCSS-A  Concerns 

An  HQDA  G-4  manager  noted  that,  for  an  ERP,  it  is  important  to  define 
the  enterprise  and  then  set  up  business  rules  for  enterprise  requirements,  but 
existing  policies  were  developed  with  a  narrower,  tactical  focus  rather  than  a 
broader,  strategic  focus.  His  point  echoes  that  of  an  earlier  GAO  report 
(Williams,  Rhodes,  and  Solis,  2007),  which  concluded  that  the  Army  did  not 
have  an  enterprise  view  when  determining  changes  needed  in  its  business 
processes.  Specifically,  the  authors  stated  (p.  6)  that 

the  Army’s  existing  strategy  perpetuates  some  of  the  cumbersome 
and  ineffective  business  processes  that  are  currently  used  in  its 
existing  legacy  system  environment.  The  benefits  of  an  ERP  solution 
include  streamlining  of  business  processes  and  elimination  of  data 
redundancy. 

While  GCSS-A  promises  to  capture  needed  field-level  maintenance  data  that 
are  currently  missing  or  low  quality,  doing  so  will  require  changes  in 
maintenance  reporting  practices. 

At  present,  it  is  not  possible  to  evaluate  GCSS-A  data  quality  in  an 
operational  environment.  An  interviewee  questioned  whether  the  planned  data 
cleansing,  which  is  to  precede  the  transfer  of  data  from  legacy  systems  to 
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GCSS-A,  will  be  sufficient — or  whether  repeated  data  cleansings  will  be 
needed.  Business  process  changes,  such  as  less  manual  data  entry,  may  reduce 
the  need  for  subsequent  data  cleansings. 

LMP  Concerns 

Interviewee  statements  also  revealed  LMP  limitations  with  respect  to  LCS 
data.  According  to  a  CECOM  manager  familiar  with  LMP,  the  system  does  not 
track  depot  maintenance  by  end  item  serial  number,  and  the  system  therefore 
does  not  address  the  need  for  serial-level  renewal  data.  Also,  because  the  transition 
to  LMP  has  been  difficult,  many  have  not  yet  experienced  the  intended  benefits 
of  the  system.  Examples  of  implementation  difficulties  appear  in  Table  2.4. 

The  Army  is  working  to  apply  lessons  learned  in  LMP  implementation  to 
date  and  a  recent  GAO  report  (Solis,  Khan,  and  Barkakati,  2009:7)  concluded 
that  the  Army  has  begun  to  implement  most  GAO  recommendations  related  to 
LMP.  Still,  as  with  GCSS-A,  the  ultimate  impact  of  LMP  on  LCS  data  cannot 
yet  be  determined. 


Table  2.4 

Interviewee  Statements:  Experiences  with  LMP 


Interviewee 

Statement 

CCAD 

manager 

"We're  okay — producing.  It  had  an  impact  on  productivity;  we  saw 
productivity  dip.  We  had  to  change  our  whole  business  process.  LMP  doesn't 
calculate  revenue  from  a  part  until  it  hits  the  floor.  It  won't  hit  the  assembly 
phase  for  200  days,  so  LMP  won't  recognize  the  revenue.  We  had  revenue 
below  targets  and  took  a  beating  at  AMC;  they  didn't  understand  how  LMP 
was  changing  the  calculation." 

CECOM 

manager 

"We  need  to  do  a  great  job  of  data  cleansing  because,  once  it's  in  there, 
[LMP]  is  unforgiving." 

CECOM 

manager 

"LMP  started  in  July  2003,  and  now  it's  2010  and  we  still  haven't  fielded  it 
to  TACOM  because  of  all  the  issues  up  until  now.  They  were  supposed  to 
start  with  the  other  LCMCs  a  year  and  a  half  afterwards,  but  nobody  else 
would  take  it.  There  have  been  thousands  of  change  requests — problems 
people  need  resolved.  They've  resolved  a  lot,  but  we  keep  adding  [new 
change  requests]  to  it." 
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Item  Unique  Identification  (IUID)  May  Help  Improve 
Data  Quality ,  but  Impact  May  Depend  on 
Implementation  Resources  and  Scope 

•  IUID  -  marking  items  with  unique  item  identifiers  in  machine-readable  code 

•  Purposes  (DoD  5000.02,  2008:79:  HQDA  Implementation  Plan  for  Serialized  Item  Management  (SIM),  2008) 

-  Link  pedigree,  usage,  maintenance,  warranty,  status,  and 
configuration  data  to  equipment 

-  Enhance  LCM 

-  Provide  more  accurate  asset  valuation 

•  Concerns: 

-  IUID  is  an  unfunded  OSD  requirement 

-  Potential  for  variation  in  implementation 

•  PM  has  ultimate  decisionmaking  authority  over  which  items  are 
selected  for  IUID,  in  accordance  with  HQDA  and  DoD  guidance 
(items  >$5,000,  serially  tracked,  or  deemed  necessary  by  the  PM) 

•  Legacy  MIS  will  be  “modernized  on  a  case-by-case  basis” 

•  Distribution  of  scanners  is  to  be  determined 
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Another  major  data  initiative  in  the  DoD  and  the  Army  is  Item  Unique 
Identification  (IUID),  a  program  to  mark  items  with  unique  item  identifiers 
(UIIs),  machine-readable  codes  that  distinguish  each  item  from  other  items  and 
do  not  change  during  the  life  of  an  item  (Deputy  Assistant  Secretary  of  the 
Army,  Acquisition  Policy  and  Logistics  (SAAL-ZL),  2009).  The  IUID 
implementation  plan  (SAAL-ZL,  2009)  specifies  that  the  UII  “shall  be  used 
globally  as  the  common  data  key  in  financial,  property  accountability, 
acquisition,  supply,  maintenance,  and  logistics  automated  information  systems 
(AIS).”  Goals  of  the  program  include  more  precise  data  capture,  increased  asset 
visibility,  improved  management  of  historical  data,  a  better  ability  to  determine 
property  values  in  financial  audits,  and  better  logistics  support.  The  plan  also 
specifies  that  all  Class  VII  items  (major  end  items)  should  be  marked  by 
December  2010  (SAAL-ZL,  2009:16). 
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The  extent  to  which  these  goals  are  achieved,  however,  depends  on 
implementation  resources  and  scope.  A  LOGSA  IUID  subject  matter  expert 
pointed  out  that  IUID  is  an  unfunded  OSD  requirement.  Also,  a  PM  Stryker 
manager  noted,  “IUID  is  being  mandated  across  DoD,  but  funding 
implementation  is  being  left  up  to  the  PMs.  For  it  to  be  successful,  all  the 
systems  need  to  be  able  to  pass  information  from  point  to  point.”  The  HQDA 
and  DoD  guidance  states  that  items  selected  for  IUID  should  be  those  that  are 

•  greater  than  $5,000; 

•  serially  tracked;  or 

•  deemed  necessary  by  the  PM. 

The  Army  IUID  strategy  document  (SAAL-ZL,  2008:9)  states  that  the  “PM  is 
the  ultimate  decision  making  authority  regarding  which  items  are  marked 
consistent  with  DoD,  HQDA,  and  PEO  guidance.” 


-73- 


3.  Overall  Findings  and  Recommendations 


Summary  of  Principal  Information  System  Gaps  and 

Contributing  Factors 

•  LCS  databases  are  not  designed  for  ease  of  access  and  use  by  Army 
analysts 

-  Multiple,  sometimes  lengthy  approval  processes 

-  Insufficient  flexibility 

-  Data  visibility  hampered  by  dispersion  in  multiple  systems  and  by  partitioning 
within  systems 

•  LCS  data  have  substantial  quality  (accuracy  and  completeness)  problems 

-  Serial  number  discrepancies 

-  Inconsistent  age  data 

-  Poor  usage  data  capture 

•  Some  LCS  data  important  to  analyses  are  not  captured  or  archived 
sufficiently 

-  Most  unit  maintenance  and  scheduled  maintenance  actions 

-  Renewal  actions 

•  Policy,  design,  and  execution  factors  contribute  to  data  problems 

-  Policies  do  not  require  work  orders  for  all  organizational  maintenance;  do  not 
require  STAMIS  archiving  of  renewal  data;  rely  on  units  to  check  data  quality 

-  Systems  still  depend  on  manual  input  and  have  limited  error-  and  mistake¬ 
proofing 

-  Lack  of  emphasis  on  and  enforcement  of  data  reporting  by  units  and  depots 

RAND  LC  Sustainment  Data  39  01/10 


The  slide  above  outlines  the  STAMIS  LCS  data  limitations  and 
contributing  factors  described  in  the  previous  section  of  this  document.  First, 
Army  analysts’  access  and  use  of  LCS  data  are  hindered  by  lengthy  approval 
processes  for  accounts  on  information  systems;  insufficient  flexibility  of  the 
systems;  and  the  dispersion  of  data  in  multiple  systems  as  well  as  restrictions 
that  allow  users  to  access  only  a  portion  of  the  data  in  such  systems.  Second, 
there  are  substantial  data  quality  issues,  particularly 

•  serial  number  errors  and  discrepancies  across  data  sources; 
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•  inaccurate  manufacture  dates  (or  inconsistent  treatment  of  age — 
sometimes  based  on  year  of  recapitalization  and  sometimes  based  on 
manufacture  date);  and 

•  many  missing  and  implausible  odometer  readings. 

Third,  STAMIS  are  not  capturing 

•  most  organizational-level  maintenance  (i.e.,  only  capturing  NMC 
repairs  at  the  organizational -level);  and 

•  most  reset,  refurbishment,  recapitalization,  and  overhaul  actions  by 
end  item  serial  number. 

Finally,  multiple  policy,  design,  and  execution  factors  may  be  contributing  to 
these  problems  as  listed  in  the  above  summary. 
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Recommendation:  Take  Actions  to  Improve  Access, 
Ease  of  Use,  and  Quality  of  LCS  Data 

•  Improve  access  and  ease  of  use  of  LCS  data  for  analysts 

-  Single,  secure  source  of  data 

-  One-time  approval  process 

-  Increase  transparency  and  flexibility  of  database 
applications 

•  Error-proofing  and  mistake-proofing  to  improve  quality  of  LCS 
data 

-  Single  data  entry,  automated  when  possible,  for  serial 
numbers  and  manufacture  dates 

-  Embedded  data-harvesting  instruments 

-  Immediate  rejection  of  data-entry  errors  in  electronic  forms 
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A  number  of  opportunities  exist  for  improving  the  ease-of- access/ use, 
quality,  and  historical  span  (archiving)  of  Army  LCS  data.  To  increase  Army 
analysts’  awareness  of — and  access  to — STAMIS  LCS  data,  greater 
centralization  of  the  data  is  needed.  Further  expansion  of  LIW  to  include 
AWPS,  SDS,  MMIS,  and  AEPS  data  would  help  accomplish  this.  Also,  to 
reduce  the  number  of  lengthy  application  and  approval  processes  needed  for 
data  access,  the  Army  could  consider  establishing  a  list  of  authorized  users  for 
unclassified  systems.  The  account  approval  process  could  consist  of  verifying 
that  an  applicant  is  on  such  a  list.  Additionally,  information  system  users  on  the 
list  could  automatically  be  granted  full  access  to  all  unclassified  STAMIS  data 
in  an  information  system  rather  than  partial  access  (i.e.,  being  limited  to 
specific  modules). 

To  increase  the  usability  of  LCS  data  sources,  several  information  system 
design  changes  may  be  helpful.  In  particular,  users  should  have  the  option  of 
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extracting  multiple  years  of  data,  rather  than  being  limited  to  one  month  or  one 
year  at  a  time.  Along  the  same  lines,  query  options  should  be  increased.  The 
flexibility  to  specify  the  geographic  region,  level  of  analysis  (serial,  unit,  or 
higher),  and  number  of  variants  (one  NIIN  versus  all  NIINs  with  the  same 
FSC)  will  allow  data  to  more  easily  serve  a  wider  variety  of  analytical  purposes. 
Additionally,  greater  information  system  flexibility  will  reduce  the  burden  on 
the  user,  such  as  the  need  to  run  a  report  repeatedly  and  write  programs  to  link 
piecemeal  data,  which  should  increase  the  number  of  users  with  the  time  and 
capabilities  to  use  the  data.  LIDB  offers  many  options  for  extracting  data,  but 
other  systems  (ILAP,  OSMIS,  PBUSE)  are  less  flexible.  Other  possible  steps 
toward  improved  usability  include  improving  navigation  tools  and 
reconsidering  the  arrangement  of  modules  in  LIW,  and  improving  user  support 
in  OSMIS  (by  developing  a  detailed  OSMIS  manual,  for  instance).  Such 
actions  are  likely  to  increase  database  transparency  to  Army  analysts. 

To  improve  data  quality,  it  is  important  to  increase  error-proofing  and 
mistake-proofing.  A  critical  error-proofing  step  is  reducing  manual  input  of 
data.  A  one-time  input  of  a  vehicle’s  serial  number  and  manufacture  date,  with 
the  single  input  populating  the  serial  number  field  in  multiple  systems,  would 
reduce  the  opportunity  for  data  entry  errors.  This  single  data  entry  could  occur 
when  the  Army  first  receives  a  vehicle  from  a  manufacturer.  Then  all  vehicles 
would  have  only  one  number  listed  in  PBUSE,  and  that  number  would  drive 
the  serial  numbers  in  usage,  readiness,  and  maintenance  systems. 

In  addition,  embedding  data-harvesting  instruments  in  new  vehicles — 
smart  cards  attached  to  vehicle  onboard  diagnostic  ports — would  improve  the 
capture  and  quality  of  vehicle  usage  and  performance  data.  As  one  HQDA  G-4 
manager  noted, 

As  long  as  we  have  soldiers  input  data,  we’re  going  to  have  accuracy 
problems  ...  I’d  like  to  see  an  interrogator  at  the  front  gate  of  a 
[forward  operating  base)  (to  measure  odometer,  fuel,  hours  since 
[the  vehicle]  last  left  the  gate)  so  that  all  that  would  be  collected  in 
an  automated  fashion  rather  than  having  a  soldier  fill  out  a  5990 
and  have  a  dispatcher  who  fat  fingers  it  into  the  system. 
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In  a  similar  vein,  a  PM  Stryker  manager  commented, 

One  of  the  big  problems  we  have  is  that  there’s  no  automated 
barcoding  system  that  works  across  the  board — where  you  are 
scanning  a  system  and  not  looking  for  human  interaction  [and] 
where  all  the  systems  talk  to  each  other.  There’s  no  system  that 
tracks  from  the  depot,  to  shipping,  to  picking  up  at  the  post  or  unit 
level.  That  would  really  be  a  good  thing  for  parts  .  .  .  for  vehicles, 
too.  Anything  with  manual  intervention  does  come  down  to 
mistakes. 

As  with  error-proofing,  designing  more  mistake-proofing  into 
information  systems  would  enhance  data  quality.  ULLS-G/AMSS  could  either 
completely  reject  odometer  readings  that  have  digits  after  a  decimal  point — and 
provide  the  reason  for  rejecting  them — or  incorporate  software  changes  that 
allow  it  to  interpret  odometer  readings  with  decimal  points  correctly. 
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Recommendation:  Ensure  Collection  and  Archiving 
of  All  LCS  Data  Needed  for  Analysis 

•  Policy: 

-  Include  analysts  in  requirements  development  process  for  new 
information  systems 

-  Include  requirements  for  data  elements  in  Army  policy 

-  Establish  a  renewal  maintenance  archiving  standard  for  PMs,  with  central 
record  maintained  in  LIW 

•  Ensure  this  includes  contractor  data 

•  Design: 

-  Use  list  of  required  data  elements  to  identify  gaps  in  GCSS-A  and  LMP 

-  Convert  ILAP  to  two-level  maintenance 

-  Ensure  SAMS-E  work  orders  can  be  opened  for  all  scheduled  and 
unscheduled  maintenance 

•  Execution: 

-  Enforce  LCS  data  reporting  with  automated  reminders  and  feedback 

-  Communicate  to  operational  Army  (those  responsible  for  executing 
policies  and  gathering  data)  how  the  data  will  help  them  in  the  long  run 
(Galway  and  Hanks,  1996) 

•  Balance  burden  to  the  field  with  data  benefits 
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Just  as  there  are  opportunities  for  increasing  the  access,  use,  and  quality  of 
LCS  data,  there  are  multiple  avenues  toward  more  complete  data  capture 
overall.  First,  it  is  likely  that  Army  policy  changes  will  be  needed  to  ensure 
better  alignment  of  data  policies  with  strategic  decisionmaking  and  analytical 
needs.  Army  policies  should  specify  all  data  elements  that  STAMIS  need  to 
capture.  Such  changes  could  emerge  from  a  policy  review  and  revision  process 
that  includes  analysts  and  that  considers  data  elements  discussed  in  this  study. 
Establishing  a  standard  for  archiving  renewal  data  by  serial  number  would  be 
particularly  valuable. 

Second,  changes  in  current  and  future  STAMIS  designs  warrant 
consideration.  An  in-depth  examination  of  GCSS-A  and  LMP  may  reveal 
modifications  needed  to  ensure  that  SALE  provides  all  critical  LCS  data 
elements.  In  addition,  options  for  converting  ILAP  to  two-level  maintenance 
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should  be  investigated,14  as  should  software  changes  allowing  multiple  SAMS-E 
work  orders  to  be  open  at  once  for  a  given  serial  number. 

Third,  execution  of  Army  data  policies  could  be  improved  through 
automated  reminders  to  units,  like  monthly  reminders  to  report  readiness,  or 
reminders  to  conduct  and  open  work  orders  for  scheduled  services  on  vehicles. 
Feedback  after  a  unit  provides  reports  (or  does  not  provide  them)  may  reinforce 
the  idea  that  usage  and  readiness  reporting  are  required  and  that  unit 
compliance  is  being  monitored. 

An  earlier  study  of  Army  data  quality  (Galway  and  Hanks,  1996)  noted 
that  a  key  to  better  execution  of  data  policies  may  be  ensuring  that 
organizations  that  generate  the  data  understand  how  it  will  be  used.  More 
specifically,  Galway  and  Hanks  (1996:15)  observed  that  the  persistence  of 
missing,  invalid,  and  inaccurate  data  over  time  can  be  a  symptom  of  a 
disconnect — a  difference  in  the  perspective  of  the  organization  that  collects  data 
and  the  organization  that  uses  the  data: 

This  can  happen  when  data  users  and  creators  are  in  different  parts 
of  an  organization  or  in  completely  different  organizations.  In  this 
case,  there  may  be  no  adjudicator  who  can  balance  data  burden  (for 
the  creators)  with  data  benefits  (for  the  users). 

Thus,  a  vital  step  toward  higher  data  quality  is  to  communicate  to  the 
operational  Army — the  units  responsible  for  executing  policies  on  data 
collection — how  better  data  will  help  them,  and  the  overall  Army,  in  the  long 
run.  Additionally,  there  will  need  to  be  negotiation  and  “agreement  within  or 
between  organizations  as  to  what  data  are  required,  what  is  acceptable  data 
quality,  and  how  costs  are  to  be  allocated  and  benefits  shared”  (Galway  and 
Hanks,  1996:  15). 


14  For  example,  to  capture  field-level  maintenance,  one  option  may  be  to  offer  three 
types  of  reports:  organizational  and  DS  reports  for  time  periods  preceding  two-level 
maintenance  and  field-level  maintenance  reports  for  more  recent  time  periods. 
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In  summary,  the  Army  has  been  facing  data  challenges  for  some  time  and 
has  been  actively  pursuing  STAMIS  changes  to  address  them.  However,  a 
current  assessment  of  LCS  data  issues,  causes,  and  opportunities  was  needed  to 
inform  ongoing  and  future  improvement  efforts.  This  study  aimed  to  provide 
such  an  assessment.  Our  findings  highlight  a  range  of  data  access,  quality,  and 
breadth  issues  that  merit  attention  and  also  suggest  that  a  combination  of  policy 
revisions,  information  system  design  changes,  and  better  policy  execution  are 
critical  to  addressing  such  data  issues.  Taking  a  comprehensive  approach  to 
LCS  data  improvement  will  help  ensure  that  Army  managers  and  analysts  have 
the  information  needed  to  manage  equipment  life  cycles  effectively. 


-81  - 


Appendix  A: 

Reports  of  LCS  Data  Problems  Hindering 

LCM  Decisions 


The  following  tables  list  studies  that  discuss  effects  of  data  gaps  on 
military  and  commercial  fleets.  Organizations  in  the  studies  experienced 
problems  such  as  inaccurate  reports  of  reset  completions,  unclear  data  on  reset 
funding  obligations,  low-quality  data  on  equipment  utilization,  and  failure  to 
retain  baseline  cost-performance  analyses  (to  compare  actual  and  expected 
performance)  for  contractor  logistics  support.  The  right-hand  column  of  each 
table  lists  (in  red)  LCM  decisions  and  analyses  hindered  by  these  data 
problems. 


Prior  Studies  Have  Reported  Cases  of  LCS  Data  Problems 
Hindering  Key  LCM  Analyses  and  Decisions 


Report 

Fleets 

Data  Problems 

Hindered  LCM  Decisions 
and  Analyses 

AAA  2008 

Army  fleets 

Funding  obligations 
for  reset 

Reset  completion 
data 

-  How  many  systems  are 
available  to  distribute  to  units? 

-  How  much  to  budget  for  reset 

-  How  many  systems  have 
washed  out? 

GAO-09-865 

Army  and 
Marine  Corps 
Fleets 

Workload  forecasts 

-  Determining  depot  capabilities 
needed  for  future  workloads 

-  What  depot  resources  are 
needed  to  support  new  and 
modified  weapon  systems? 

Congressional 
Research 
Service  2006 

Military 

equipment 

Condition  and 
disposition  data 

-  Which  equipment  should  be 
disposed  of,  and  how? 

GAO-06-141 

Military  and 
civilian 
ground  fleets 

Usage  data 

-  Whether  to  downsize  fleet 
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Prior  Studies  Have  Reported  Cases  of  LCS  Data  Problems 
Hindering  Key  LCM  Analyses  and  Decisions 


Report 

Fleets 

Data  Problems 

Hindered  LCM 

Decisions  and  Analyses 

Saltzgiver 

2004 

Commercial 
ground  fleets 

Utilization  data 
(mileage) 

-  What  is  cost/per  mile  to  operate? 

-  Are  vehicles  utilized  appropriately? 

-  Are  continuous  improvement 
efforts  effective? 

-  What  preventive  maintenance 
intervals  are  appropriate? 

-  Which  vehicles  should  be  disposed 
of? 

GAO-04-645 

Federal 

civilian 

agency 

aircraft 

Number  of  aircraft 
Usage  data 

Support  cost  data 

-  How  much  to  budget  for  operations 

-  How  much  to  budget  for 
maintenance 

-  Whether  to  dispose 

GAO-02-306 

Army  and 

Navy  weapon 
systems 

Not  retaining 
baseline 
estimates  of 
contractor 
logistics  support 
cost-effectiveness 

-  Are  initial  cost  estimates  for  CLS 
are  being  achieved? 

-  Are  adjustments  in  CLS  needed? 

-  Should  the  military  continue 
outsourcing  logistics  support 
for  a  system? 
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Appendix  B: 
Interview  Questions 


We  Asked  Subject  Matter  Experts  About 
Experiences  with  LCS  Data  in  the  Army 


Topic 

Questions 

Background 

What  is  your  role  in  your  organization,  and  what  types  of  equipment 
sustainment  data  do  you  use  for  your  work? 

How  do  you  normally  obtain  these  data  (from  what  sources)? 

Uses  for  life  cycle 
sustainment  data 

For  what  types  of  tasks  (e.g.,  analyses,  plans,  or  decisions)  are  such 
data  needed? 

Data  limitations 

Do  the  data  that  you  use  have  any  limitations  (e.g.,  quality,  breadth, 
accessibility)  that  you  would  like  to  see  addressed?  If  so,  please 
describe. 

Impact  of  data 
limitations 

What  Is  the  impact  on  your  work,  your  colleagues'  work,  or  the  overall 
organization  of  these  data  limitations? 

(Can  you  provide  examples  of  plans,  decisions,  or  analyses  that  were  or 
adversely  affected  by  sustainment  data  constraints?) 

Factors  affecting 
data  breadth,  quality, 
accessibility 

What  factors  (e.g.,  human  factors,  organizational  processes, 
organizational  structures)  may  be  contributing  to  data  problems? 

Do  you  anticipate  that  any  planned  changes  (e.g.,  in  systems, 
organizations,  or  policy)  will  affect  the  breadth  or  quality  of  sustainment 
data  available?  If  so,  please  describe. 
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Appendix  C: 

Standard  Sources  of  Army  LCS  Data 


The  tables  below  list  the  sources  of  STAMIS  data  examined  by  the 
research  team.  We  obtained  accounts  to  access  the  information  systems  directly, 
became  familiar  with  each,  ran  reports,  and  evaluated  both  the  data  source  and 
the  data  extracted. 


Standard  Army  Sources  of  Life  Cycle  Sustainment  Data 


Database 

Purpose 

AEPS  (Army  Electronic 
Product  Support) 

AMC  Logistics  Web  Portal  to  provide  various  types  of 
supply,  maintenance,  technical,  and  acquisition  logistics 
information 

AWPS  (Army  Workload 
and  Performance  System) 

System  for  managing  maintenance  workload,  managing 
maintenance  resources,  and  tracking  performance  of  depots 
and  other  facilities 

FEDLOG  (Federal 

Logistics  Database) 

Logistics  information  product,  published  by  Defense 

Logistics  Information  Service  (DLIS),  with  cataloging 
information  for  stock  numbers  and  part  numbers 

ILAP  (Integrated  Logistics 
Analysis  Program) 

System  providing  supply,  maintenance,  and  financial 
management  reports 

LIDB  (Logistics  Integrated 
Database) 

Database  with  multiple  logistics  reports,  including 

•  Serial  number  usage  report:  odometer  readings  for 
usage-reportable  equipment 

•  Ground  equipment  tracker:  year  of  manufacture  from 

TEDB  (The  Army  Maintenance  Management  System 
(TAMMS)  Equipment  Data  Base) 
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Standard  Army  Sources  of  Life  Cycle  Sustainment  Data 


Database 

Purpose 

MMIS  (Modification 
Management  Information 
System) 

HQDA  G-4  system  to  track  modification  work  orders 
(MWO)  that  are  applicable  to  an  item,  have  been  applied, 
and  are  scheduled  to  be  applied. 

OSMIS  (Operating  and 

Support  Management 
Information  System) 

SAFM-CE  system  tracking  operating  and  support 
information  for  Army  weapon/materiel  systems 

PBUSE  (Property  Book  Unit 
Supply  Enhanced) 

HQDA  G-4  system  for  property  accountability 

RIDB  (Readiness  Integrated 
Database) 

Database  tracking  current  and  historical  availability  of 
readiness-related  equipment 

SLAMIS  (Standard  Study 
Number-Line  Item  Number 
Automated  Management  and 
Integrating  System  (SLAMIS)) 

Application  to  provide  access  to  “chain  of  custody"  data 
relationships  and  tools  to  support  LCM  of  major  end  items. 
Includes  a  Nonstandard  Line  Item  Number  (NSLIN) 
catalog  with  unit  price,  manufacturer,  supply  class,  and 
other  information  about  NSLINs. 
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Appendix  D: 

Data  Issues:  Bases  of  Medium  and  Low  Ratings 

of  Data  Elements 


The  slides  in  this  Appendix  describe  the  basis  for  our  earlier  “M”  and  “L” 
ratings  of  demographic,  operations,  maintenance,  and  disposal  LCS  data 
elements  in  Army  STAMIS. 


Issues  with  Demographic  Data  on  Army  Equipment 


Data 

Element 

Data 

Source 

Ease  of  Use/ 
Access  Issues 

Quality  Issues 

History 

Issues 

Serial 

Number 

ILAP 

LIDB 

PBUSE 

PBUSE  access 
requires  special 
permissions  that 
can  be  difficult  to 
obtain. 

All  3  databases  contain  many  serial  number  errors: 
typos  that  cause  the  same  serial  number  to  look 
like  two  different  numbers,  serial  number  and 
registration  numbers  reversed,  unit  notes  in 
registration  number  or  serial  number  field  (in 
PBUSE). 

NSN 

NSLIN 

LIDB 

SLAMIS 

Some  NSNs  are  not  accurate  (e.g„  do  not 
correspond  to  the  serial  number  sequence  or  are 
not  consistent-tor  the  same  vehicle-in  usage  and 
maintenance  data  sources) 

NSLIN  corresponds  to  wide  variety  of  systems  with 
wide  price  range 

Age 

LIDB/ 

TAMMS 

Many  fleet  serial  numbers  lack  manufacture  dates 
in  TAMMS.  Some  manufacture  dates  do  not 
correspond  to  the  vehicle  type  ;  they  instead 
correspond  to  an  older  variant. 

Potential  Contributing  Factors 

•  Information  system  design:  manual  data  entry  and  quality  checks;  inconsistent  serial  number  field  lengths 
in  maintenance  and  supply  databases;  multiple  prices  allowed  for  same  NSLIN;  recapitalization  date  can 
overwrite  original  year  of  manufacture 

•  Policy  execution:  Insufficient  training  and  enforcement  of  policy  to  retain  serial  number  throughout  item 
life  cycle 
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Issues  with  Demographic  Data  on  Army  Equipment  (cont’d) 


Data 

Element 

Data 

Source 

Ease  of  Use/ 
Access  Issues 

Quality  Issues 

History  Issues 

Weight/ 

Cube/Cost 

FEDLOG 

Weight/cube  data  not  available  for 
many  class  7  items  and  are 
questionable  for  many  class  9  items. 

NA  for  weight  and  cube  data. 
Historical  price  data  are  not 
available. 

Unit 

LIDB 

PBUSE 

PBUSE  access 
requires  special 
permissions  that  can  be 
difficult  to  obtain. 

In  both  systems  there  are  invalid  and 
missing  UlCs.  ARCENT  Asset 

Visibility  team  indicated  that  forward 
detachments  should  have  UlCs  ending 
in  0  or  1  -  but  that  some  do  not. 

ARCENT  asset  visibility  team 
indicated  it  is  not  possible  to 
determine  when  a  TPE  item 
arrived  in  theater  and  track 
where  it  moved  over  time. 

In  PBUSE.  one  can  only  tell  if 
an  item  is  currently  in  theater, 
not  when  it  previously  was  in 
theater. 

Location 

LIDB 

PBUSE 

PBUSE  does  not 
appear  to  offer  a  query 
by  location  or  standard 
geographic  region. 

In  LIDB  the  translation  of  UIC  is  non¬ 
standard:  same  UIC  can  have  multiple 
translations. 

PBUSE  consolidated  property  listing 
has  no  translation  of  UIC. 

Potential  Contributing  Factors 

•  Information  System:  Design:  limited  error  &  mistake-proofing  (dependence  on  manual  detection/ 
correction  via  "AMDF  Discrepancy  Report"  or  'Price  Challenge  Form"  on  LIW);  PBUSE  queries  designed 
for  field  users  rather  than  analysts. 

•  Policy:  PAM  700-32  allows  multiple  data  entry  points:  When  weight/cube  data  not  in  FEDLOG,  Army 
installations  can  develop  such  data,  which  may  be  submitted  to  the  Army  Master  Data  File  (AMDF,  part  of 
FEDLOG). 

•  Policy  execution:  Insufficient  training  and  enforcement  of  policy  on  UlCs  for  forward  detachments. 
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Issues  with  Operations  Data  on  Army  Equipment 


Data 

Element 

Data 

Source 

Ease  of  Use/ Access  Issues 

Quality  Issues 

History  Issues 

Readiness 

RIDB 

RIDB  covers  a  low  percentage  of 
fleets. 

Usage: 

Mileage/ 

Operating 

Hours 

LIDB 

OSMIS 

Location  of  usage  data  by  serial  number 
(in  MMR  Serial  Reporting  module)  is  not 
readily  apparent  in  OSMIS.  Also,  it  is 
only  possible  to  pull  2  months  of 
odometer  readings  (1  month  of  usage) 
at  a  time. 

Mileage  quality  is  poor  -  we  see 
negative  usage,  implausible 
usage,  and  many  missing 
odometer  readings,  especially  for 
deployed  units  and  equipment . 

There  are  more 
historical  data  in 

LIDB  than  in 

OSMIS. 

Usage:  Fuel 
Consump¬ 
tion 

OSMIS 

Not  by  serial  number;  not 
considered  accurate  (according 
to  TACOM  Cost  and  System 
Analysis  dept);  estimates  rather 
than  actual  values 

Usage: 

Rounds 

Fired 

AEPS 

Data  not  readily  accessible.  First  need 
to  apply  for  AEPS  access,  then  apply  for 
access  to  rounds  fired  data. 

According  to  interviewees, 
breadth  is  low  (few  systems  with 
rounds  fired  data) . 

Could  not  be 
determined:  did  not 
have  direct  access 
to  rounds  fired  data. 

Potential  Contributing  Factors 

•  Policy:  DA  PAM  750-8  fosters  dependence  on  units  for  manual  data  entry  and  checks;  policy  conveys  fuel 
reporting  as  feasible  (via  form  5987-E)  but  not  mandatory;  no  policy  prohibiting  "zeroing"  odometer  when 
vehicle  is  reset. 

•  Information  system  design:  Limited  error-  and  mistake-proofing  of  odometer  readings;  lack  of  flexibility  (in 
OSMIS)  to  pull  more  than  2  months  of  odometer  readings  (1  month  of  usage)  at  a  time. 

•  Policy  Execution:  Insufficient  training  and  enforcement  of  policy  requiring  valid  monthly  odometer  readings  and 
readiness  reporting,  even  while  units  are  deployed. 
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Issues  with  Maintenance  Data  on  Army  Equipment 


Data 

Element 

Data 

Source 

Ease  of  Use/ 
Access  Issues 

Quality  Issues 

History 

Issues 

Field-level  maintenance 
repair  time  and  dates 

ILAP 

Canned  reports  in 
ILAP  limit  query 
options.  '  Dynamic 
query'  option 
provides  more 
flexibility  for  some 
of  the  maintenance 
datasets. 

There  are  serial-level  maintenance  data  in 

ILAP,  but  work  orders  are  not  capturing  most 
unit-level  maintenance  and  scheduled 
maintenance.  Parts  used  and  cost  are  well 
captured  for  deadlining  unit-level  repairs  in 
EDA,  but  some  systems  are  not  in  EDA  or 
have  little  EDA  data.  Quality  is  better  in  more 
recent  years  than  in  earlier  years. 

About  5  to 

10  years  of 
maint.  data 
available, 
depending 
on  system. 

Field-level  maintenance 
cost:  parts  and  labor 

ILAP 

GS  repair  time  and 
dates 

ILAP 

GS  cost:  parts  and  labor 

ILAP 

Renewal  time,  dates, 
and  cost  (parts  and 
labor) 

AWPS 

&  SDS 

AWPS:  Access  to 
DMOPS  was 
approved  quickly, 
but  RRAD  data 
access  was  denied. 

Data  lack  precision  -  no  renewal  dates  or 
costs  at  serial  number  level.  TPER  data  not 
included. 

Reports 
prior  to 

2003  not 
found. 

Configuration  Mgmt. 

MMIS 

Data  capture  is  better  for  some  systems  than 
for  others  -  and  there  are  some  fields  with 
inconsistent  data  types  (e.g..  MWO  field). 

Disposal:  Est.  Cost  to 
Replace 

FEDLOG 

Missing  cost  data  on  some  fleets  (e.g.. 

MRAP) 

Disposal:  Validated 
Losses 

AEL 

Tracker 

Many  records  not  validated  and/or  have  loss 
type  missing. 

Potential  Contributing  Factors 

•  Policy:  PAM  750-8  requires  ORG  WON  for  inoperative  equipment  but  not  for  non-deadlining  repairs; 

AR  750-1  relies  heavily  on  PMs  for  managing  renewal  data  rather  than  encouraging  single  centralized  source 

•  Information  system  design:  No  separate  field  on  electronic  form  2408-9  for  renewal  dates;  no  archiving  of 
renewal  dates  by  serial  number 
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